Pharma 4.0, an application of Industry 4.0 concepts in the pharmaceutical industry, aims to enhance manufacturing efficiency, product quality, and consistency. It faces challenges in areas like artificial intelligence, material traceability, optimization, process control, cyber-physical security, and data management due to the complexity involved. However, by incorporating artificial intelligence and advanced model predictive control with robust cyber-physical security measures, predictive capabilities and product quality can be significantly improved. This work focuses on implementing seven Industry 4.0 components, including AI/ML, modeling, material traceability, optimization, advanced control, cyber-physical security, and data science, in continuous pharmaceutical manufacturing processes.
Currently, Industry 4.0 concepts are being applied to the pharmaceutical industry to achieve Pharma 4.0 paradigm. It is promoted by both industries, academia and regulators. Pharma 4.0 reduces the time and resources needed for continuous pharmaceutical manufacturing and improves product quality and production consistency. It has many advantages but also have bigger challenges on the applications of artificial intelligence (AI)/machine learning (ML), material traceability, optimization, advanced process control, cyber-physical security, and data management side because of the different levels of complexities involved. The predictive capabilities and the quality of the pharmaceutical products can be improved significantly via employing the artificial intelligence and the advanced model predictive control (MPC) system if an appropriate cyber-physical security defense is in place.
In this work, seven components of industry 4.0 namely artificial intelligence (AI)/machine learning (ML), modelling, optimization, advanced control, material traceability, cyber-physical security, and data science have been developed for the continuous pharmaceutical manufacturing process.
A continuous direct compaction tablet manufacturing pilot plant has been developed, situated at C-SOPS, Rutgers University, NJ, USA (Figure 1). This is vertical design of the plant that take advantage of gravitational material flow and therefore no convayer belt is needed. This is direct compaction line that have three levels. The feeders are placed at top level to feed API and excipients. The co-mill and blender are placed in middle level. The API and excipient streams passing through the co-mill before entering to the bledner. The co-mill is used for delumping purposes. The lubricant feeder is also placed at middle level. The lubricant is directly fed to the blender without passing through the co-mill to avoid the over lubrication. The tablet press is placed on ground floor. A chute is placed in between blender and tablet press. The PAT sensor is integrated with the plant via chute interface.. The pharma 4.0 related tools are shown in Figure 1.
There are different methods and tools useful to achieve pharma 4.0 paradigm. Artificial intelligence (AI)/machine learning (ML), modelling, optimization, advanced control, material traceability, cyber-physical security, and data science have been developed for the continuous pharmaceutical manufacturing process.
2.1. Artificiel intelligence (AI)/ machine Learning (ML)
AI/ML algorithms have been highly successful in solving complex problems for many industries to predict the process response accurately. The pharmaceutical industry has yet to see the potential impact of a machine learning based self-trained process for efficient manufacturing of high-quality products. There are different AI approaches that may have potential applications in continuous pharmaceutical manufacturing. However, the identification of best performing AI method for continuous pharmaceutical manufacturing process is still a challenging task. In this work, four machine learning (ML) models have been trained to predict the response of continuous pharmaceutical manufacturing process and the performance of these ML models has been compared. The investigated ML methods are artificial neural network (ANN), 1D convolution neural network (CNN), long short-term memory (LSTM), and random forest (RF).
Inspired by the nervous system and chemical-based neuron activation in a biological brain, artificial neural networks (ANN) are a multi-layered black-box approach to digitally recreating non-linear systems' functionality with a series of interconnected signals that get sent to neurons.
For example, given a 3-layer fully connected neural network, an array of input values X of size N, an array of output Y of size M, and four hidden layers of size H are shown in Figure 2. The output of X's input values is connected to each node in the hidden layer. Each connection between layers is multiplied by a unique weight wj and summed with a bias b resulting in:
The training process for all neural networks described in this paper follows the ADAM optimizer algorithm.
A convolutional neural network (CNN) is a class of ANN that uses convolution in place of multiplication. The traditional CNN extracts feature in a 2-Dimensional image, while the 1D-CNN approach observes an input that spans some window of time of an arbitrary length. Unlike the RNN approach, the 1D-CNN pre-processes the input data before training and does not require backward propagation. Essentially, the 1D-CNN treats the input time window as a single image of a larger dataset and aims to identify the step response of changing some input with a smaller set of data labels as output.
Long Short-Term Memory (LSTM) is a form of RNN (Recurrent neural network) that uses specialized cells with feedback connections and is used in long sequences of data such as time-series data in speech and weather forecasting or text-based image problems.
Random Forests (RF) are an approach to machine learning methods that use an ensemble of randomly generated decision trees to classify or regress by majority vote. Initially used for classification problems, random forests are effective for regressive multivariate and time-series forecasting problems. A decision tree is an intuitive set of binary conditions that branch to a result based on an input's qualities. In the context of a manufacturing process, sensor data inserted into a decision tree pass through simple conditions until the most likely result is decided.
The best performing ML model is selected for the continuous pharmaceutical tablet manufacturing process for real time prediction.
2.2. Modeling and simulation
A flowsheet model of continuous pharmaceutical tablet manufacturing process has been also developed. The heart of the flowsheet model is the unit operation model library. The mathematical model of continuous pharmaceutical unit operations such as feeder, co-mill, blender, and tablet press have been developed. The model has been simulated in gPROMS (Process Systems Enterprise), Matlab, and Python simulation platforms. These models can be used to generate an integrated process flowsheet model. The different configuration of the virtual CM line can be developed and investigated using this modeling toolbox.
2.3. Dynamic optimization
The process flowsheet model mentioned in section 3.2 has been used to optimize the continuous pharmaceutical tablet manufacturing process. For example, a systematic framework including the methods and tools has been developed for dynamic optimization of the feeder refill strategies. The deviation of the outlet mass flow of the feeder from the targeted flow rate has been minimized to obtain the optimum value of the feeder refill parameters. The material properties also affect the refill strategy meaning that the feeder refill strategy needs to be frequently optimized if there are any changes in the materials and plant. Therefore, the developed model and dynamic optimization tool can save the time and recourses as well as improve the product quality significantly.
2.4. Advanced model predictive control (MPC)
The automation and control are the key to fully utilize the benefits of the continuous pharmaceutical manufacturing process. For automation, the continuous manufacturing line has been integrated with two distributed control systems (DletaV, PCS7). There is a switch to move from one operating system to another operating system. This allows us to cover wider range of industrial continuous manufacturing system. The control panel is connected to the pilot-plant through standard communication systems. Feeders are connected through field bus device, co-mill and blender are connected through serial ports, and tablet press is connected via OLE process control (OPC). There are two control panels, one for DletaV and one for PCS7. But, both control panels are connected with the pilot-plant via same communication protocols and wirings. The control panels are connected to their respective control computers where control software (DeltaV or PCS7) is installed. Therefore, the switch is useful to move from one distributed control system to another. The implementation of the feedback control system into the plant is previously described.
In the continuous pharmaceutical manufacturing (CPM) pilot-plant, the CPP’s and CQA’s are controlled in real time. The critical control variables that have been controlled using model predictive control (MPC) system are drug concertation, powder level before tablet press, main and pre compression forces, tablet weight and hardness. A novel control strategy for powder level control in a chute placed in between blender and tablet press unit operation of continuous tablet manufacturing process has been developed, implemented and evaluated.
A residence time distribution (RTD) based control system has also been developed and implemented to the continuous pharmaceutical tablet manufacturing pilot-plant for real time diversion of out of specs tablets to assure content uniformity (CU).
2.5. Material traceability
A systematic framework including the methods and tools have been developed for material traceability. A corresponding software tool has been also developed to automate the material traceability procedure. The heart of the material traceability toolbox is the RTD model. The applications of the developed methods and tools have been demonstrated for material traceability of continuous pharmaceutical manufacturing process.
2.6. Cyber-physical security
The cyber-physical security is important for both plant and patient safety. There has been a significant progress in the area of ‘cyber security’ but ‘cyber-physical security’ is still an open area of research. Much less attention has been paid toward the cyber-physical security of continuous pharmaceutical manufacturing process. A cyber-physical attack in continuous pharmaceutical manufacturing process could alter the critical quality attributes (CQA’s) of the product. There are some commercially available tools such as SNAP7, Wireshark, and Tripwire that have been used in CM plant. The Snap7 has been used for data block monitoring, Wireshark has been used for network traffic monitoring, and Tripwire has been used to assure file integrity. A novel software tool named CPS (Cyber-physical security) has been developed and integrated to the continuous pharmaceutical manufacturing pilot-plant. The CPS provides the interface to integrate commercially available cyber-physical security tools, decode the data blocks to get the plant signal, and to provide an extra level of the protection to the plant.
2.7. Data management
The data management is critical for continuous pharmaceutical manufacturing. A commercially available data hub (OSI PI) has been used to collect, store, and manage the data generated from continuous pharmaceutical manufacturing pilot-plant in real time using industry 4.0 standard. The plant operating data is captured using commercially available distributed control system (e.g. Delta V, PCS7). Process Pulse II integrated with OLUPX has been used for managing, real time prediction of PAT data, and real time process monitoring.
The first component of industry 4.0, i.e. artificial intelligence (AI) has been used here for demonstration purposes. Similarly, the other components of industry 4.0 have been developed which is subject of future publication.
3.1. Training and verification of artificial intelligence (AI) models
The data used in this article consists of 16-time series experiments concatenated with real-time sensor data from the continuous tablet manufacturing process. The time-series experiments all use the same formulation and measure the time delay and step response due to changes in input.
Significant events of steady-state operation or missing sensor data were removed before the AI model training process to mitigate the number of duplicate data samples during training and validation, resulting in 48781 time-series measurements. The most extensive experiment used 8401 time-series measurements with four signal data variables. Figure 3 shows an example of a step response of the main compression force (MCF) and pre compression force (PCF) achieved during tablet compaction due to a shift in fill depth (FD). MCF and PCF can be measured in real time and therefore are ideal candidates to use for the comparison of the performance of different AI models.
The training process of the Neural Networks operates by sliding window. Two window sizes were used for the experimental approach to observe the benefits of varying sizes in forecasting accuracy. The consequence of increasing the size input and output is a decreased size to the overall dataset and an increased domain range required for accurate prediction.
In the first experiment, a' short window vector of size 30 is used as the input, and the predictive time-span extends to t+15 to keep the predicted complete step response due to change in input. The input size was selected due to the response delay in the dataset not being observed over 20 measurements, so we assume a delay of 30 is enough minimum context to predict a step change. In the second set of experiments, a 'long window' vector of size 180 is used, and we assume that the length of input provides full context for forecasting the system's response.
The data used for the Random Forest training process varies slightly, as the input is determined as the time-series signal measurements at time t. The target label is delayed 20 seconds from the labeled measurements, and predictions are made without MCF measurements at time t.
3.2. Comparison of different machine learning approaches for continuous pharmaceutical manufacturing process
The long window case study has been used here for the illustration of the concept. Some of the result is shown in Figure 4. Machine Learning approaches are analyzed to forecast the multivariate response during steady-state and step-change events and provide accurate prediction for real-time applications. We observed each neural network’s performance in forecasting real-time signal data in 3 groups of experiments in a continuous manufacturing plant. The first set of experimental data is used to train the models to forecast the main compression force (MCF) given fill depth and tablet thickness inputs. The next following experiment uses the same inputs and includes the pre-compression force (PCF) as well. And the final experiment involves models trained to predict the PCF.
Mean absolute error (MAE) is used as a performance indicator of AI models. MAE of DNN, CNN, and LSTM is shown in Figure 4. As shown in the Figure 4, the LSTM performed better in compared to DNN and CNN. For the target MCF prediction without PCF as input, the dense neural network and CNN performed similarly but DNN is slightly better than CNN. Meanwhile, given the PCF as input, both the DNN and CNN lost over 200% in MAE. In this case LSTM performed best followed by DNN and CNN. For the PCF prediction, all neural network models struggled to perform under 0.1 MAE, where the LSTM had the highest score at 0.1026.
Different components of industry 4.0 such as artificial intelligence (AI), optimization, advanced control, material traceability, cyber-physical security, and data science have been developed for the continuous pharmaceutical manufacturing process. Within AI models, Long Short-Term Memory (LSTM) performs better for the case studies that have been performed. Some drawbacks of relying on LSTMs are the training time and computation power needed to train LSTMs, which could be significant as the LSTM models grow in complexity. DNNs or 1D CNN can reduce these requirements with a sacrifice of training accuracy. The Random forest remains a promising alternative to neural networks for fast deployment but could compromise with the prediction accuracy. The current drawbacks of the AI approaches include the need of the substantial amounts of data to obtain a fully confident model, which requires a large amount of material and time. To overcome this limitation, a validated flowsheet model or digital twin could be used to train the AI model that can be used for real time applications.
This work is supported by the US Food and Drug Administration (FDA), through grant 5U01FD006487.
References
1. O’Connor, T. (2020). FDA Grand Rounds: Modernization of Pharmaceutical Manufacturing through the Adoption of Advanced Technology. https://www.fda.gov/science-research/fda-grand-rounds/fda-grand-rounds-modernization-pharmaceutical-manufacturing-through-adoption-advanced-technology.
2. Singh, R.; Sahay, A.; Muzzio, F.; Ierapetritou, M.; Ramachandran, R. Systematic framework for onsite design and implementation of the control system in continuous tablet manufacturing process. Computers & Chemical Engineering Journal 2014, 66, 186-200.
3. Singh, R., Muzzio, F., Ierapetritou, M., Ramachandran, R. A combined feed-forward/feed-back control system for a QbD based continuous tablet manufacturing process. PROCESSES Journal, 2015, 3, 339-356.
4. Barros, F. N., Bhaskar, A., Singh, R. A validated model for design and evaluation of control architectures for continuous tablet compaction process. Processes Journal, 2017, 5(4), 76. doi:10.3390/pr5040076
5. Bhaskar, A., Barros, F. N., Singh, R. Development and implementation of an advanced model predictive control system into continuous pharmaceutical tablet compaction process. International Journal of Pharmaceutics, 2017, 534 (1-2), 159-178.
6. Billups, M., Singh, R. (2018). Systematic framework for implementation of material traceability into continuous pharmaceutical tablet manufacturing process. Journal of Pharmaceutical Innovation. https://doi.org/10.1007/s12247-018-9362-9.
7. Singh, R. (2021). Cyber-Physical Security (CPS) Tool for Continuous Pharmaceutical Manufacturing Process. Pharma Focus Asia. Issue 44, 41-49. https://www.pharmafocusasia.com/information-technology/cyber-physical-security-tool.
8. Kingma, D. P., and Ba, J. (2014). "Adam: A Method for Stochastic Optimization." arXiv Preprint arXiv:1412.6980. https://arxiv.org/abs/1412.6980
9. Baytas, I. M., et al. (2017). "Patient Subtyping via Time-Aware Lstm Networks." In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 65–74.
10. Qiu, X.,, et al. (2017). "Oblique Random Forest Ensemble via Least Square Estimation for Time Series Forecasting." Information Sciences 420: 249–62.
11. Segal, M., and Yuanyuan X.. (2011). "Multivariate Random Forests." Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 1 (1): 80–87.
12. Singh, R. (2019). Systematic framework for implementation of RTD based control system into continuous pharmaceutical manufacturing pilot-plant. Pharma. Issue 34, 43-46.
Dr. Ravendra Singh is faculty of the Department of Chemical and Biochemical Engineering, Rutgers University, NJ, USA. He is the recipient of the prestigious EFCE Excellence Award from the European Federation of Chemical Engineering. His research focus is continuous manufacturing of drug substances and products. He is PI/Co-PI of several projects funded by FDA, NSF, and companies. He has published more than 75 papers, edited one pharmaceutical system engineering book published by Elsevier, written more than 12 book chapters, and presented at over 150 conferences. He is actively serving as a Journal editorial board member, and conference session chair.
