SPECIAL FOCUS: DRILLING REPORT

Applying a genetic neuro-model reference adaptive controller in drilling optimization

 Neural networks are used to model the rate of penetration while drilling, thus giving greater control over real-time logging tools that often require the driller to slow drilling to a specific penetration rate. 

José Ricardo P. Mendes and Tiago C. Fonseca, State University of Campinas; and Adriane B. S. Serapião, São Paulo State University
 

Motivated by rising drilling operation costs, the oil industry has shown a trend toward real-time measurements and control. In this scenario, drilling control becomes a challenging problem for the industry, especially due to the difficulty associated with parameters modeling.

One of the drillbit performance evaluators, the Rate Of Penetration (ROP), has been used as a drilling control parameter. However, relationships between operational variables affecting the ROP are complex and not easily modeled. This work presents a neuro-genetic adaptive controller to treat this problem. It is based on an “auto-regressive with extra input signals,” or ARX model and on a Genetic Algorithm (GA) to control the ROP.

INTRODUCTION

Operational costs associated with petroleum offshore drilling have risen significantly in the past few years. Deepwater reservoirs, whose exploration had been previously considered economically unfeasible, have become the target of the oil industry, pushing drilling rigs to increasing water depths. Therefore, drilling equipment technology must develop to support, for example, higher bottom-of-ocean pressures or provide efficient rig-positioning methods during drilling operations.

Furthermore, the number of operations has been stimulated by current oil prices, which motivate worldwide investments in petroleum exploration projects. For instance, driven by the high demand, the daily rental rates on drilling rigs doubled over a 1½-year period beginning in 2004.

As a result of the increased cost, the oil industry has leaned toward real-time measurements and control. Digital fields have been widely used in current works to denote continuous optimization of oil production, with online control techniques gaining attention in drilling operations as well. For several years, vertical-well architectures did not require complex techniques, but current horizontal, high-extend architectures require refined drilling control. Operational limits ought to be adopted in several situations, such as the use of formation logging tools that entail lower rates of penetration to correctly acquire information. Operating near these limits should provide better financial results, and therefore ROP controlling has been followed by researchers as a means to reduce drilling costs.

System identification becomes the first challenge if control is intended. The difficulty of modeling the ROP lies in its dependence on a wide range of drilling parameters, such as Weight On Bit (WOB), drillstring rotation (N), bit diameter, pore pressure gradient, equivalent circulating density and fractional tooth dullness, among others. A study examining different ROP mathematical models found that most of them use the weight on bit, rotation and the bit diameter-linearly or exponentially modified by constants-as inputs.¹ The model proposed by Bourgoyne and Young, however, is the most widely accepted one.² Some of the parameters they use are not often easily accessible, which makes their model applicable only when a very wide range of information regarding the well is available.

Neural Networks (NN) have been applied in a variety of scientific areas in the past decades. In the petroleum industry, some applications of NNs include reservoir characterization, risk assessment and drilling dynamics control.

An important characteristic regarding NNs is their ability to generalize a result based only on the knowledge contained in a data set. Because of that, they have been used to treat system identification problems such as the one focused herein, which is closely connected to finding a trustworthy ROP model. Since it involves such complex parametric relationships, using field data to train an NN provided an appropriate solution.

Bilgesu et al. have applied neural networks to predict the ROP in petroleum wells.³ They proposed a one-hidden-layer, feed-forward, back-propagation network to model the ROP, obtaining high correlation factors ranging from 0.902 to 0.982. However, Bilgesu et al. split training and testing sets randomly, picking 10% of the total data to train the network. That way, they did not separate data from one or several wells not used during training to test the NN, which would validate the network to be applied to predict the ROP of new future wells, for instance.

The present work consists of an approach to treat the system identification problem through the application of an “auto-regressive with extra input signals neural network,” or ARX NN, to create the plant model, which was then used in a Genetic Neuro-Model Reference Adaptive Controller (GNMRAC).⁴ The proposed controller constantly uses online feedback information from the plant to adjust the neuro-based model and then find the most suitable settings for the drilling parameters. Real data from seven wells drilled in an offshore field were used to train and test the NN used as the plant model. The data was divided into seven groups, each representing one of the wells. One of the wells was then used to train a network, which was then tested with the information from the other six wells.

The trained ARX NN model is used to find the fitness of each “individual” that will be used in the GNMRAC, and a genetic algorithm is employed as a global search mechanism to provide control over the ROP.

BASIC CONCEPTS OF DRILLING ENGINEERING

During the drilling operation, redundant sets of sensors located around the rig collect drilling parameter information. When a drilling platform is rented, it comes with sensors of its own. However, a mud-logging company is usually hired to make the same measurements. Some of the parameters observed by the mud-logging service include: Measured Depth (MD), True Vertical Depth (TVD), Weight On Hook (WOH), drilling pressures, torque, Weight On Bit (WOB), ROP and rotation.

Some data is taken directly from sensors installed in the rigs, such as the pressures obtained from manometers coupled to the flowlines. Other parameters, such as the rate of penetration, are calculated from other measured variables.

As mentioned earlier, several works intended to model the penetration rate mathematically. Among the main works, the rotation and the WOB, along with the bit diameter, are used as parameters to model the ROP. Rock properties are also used in some models. However, most mathematical models demand a wide range of field data. Some parameters, such as bit wear, are subject to human expertise knowledge, and thus demand either a precise training process or cautious formal documentation.

When drilling data is available, the bit-rock interaction is implicitly represented. Computational intelligence techniques have proven their capability to extract this kind information and correctly model complex parameters such as the ROP.

System identification is the effort of inferring a mathematical characterization, or a model, of a dynamic system. The methodology presented herein is based on a neural network’s modeling for the ROP. In order to better aid the driller, the NN’s goal is to predict the ROP with p steps of advance. In order to perform the system identification, the ARX NN is placed in parallel with the plant, Fig. 1.

The inputs for the plant are given by u₁(t) = TVD(t), for true vertical depth; u₂(t) = N(t), for the rotation of the drill-string; and by u₃(t) = WOB(t), for the weight on bit. The plant output is y(t) = ROP(t), for the rate of penetration. As inputs for the ARX NN besides the ones used by the plant, the plant output is used. This value is used during training after a time delay is applied, yielding the dynamic characteristic of the network. The NN then provides an output of an expected value for the ROP, or . The following section presents some basic concepts of ARX models and demonstrates the network architecture.

ARX NN ARCHITECTURE

Artificial neural networks have been used to treat several modeling-related engineering problems, mainly for their capacity of storing and using experimental knowledge acquired from data sets. To engage that, an NN must experience a learning process, through which it is exposed to some type of information and internally represents its structure for further use. As it is a widespread methodology in modern science, the basic concepts of artificial neural networks will not be dealt with in the present work. Haykin presents more detailed information about neural networks fundamentals.⁵

The ROP modeling problem was treated in a way that the information would be given in advance. The NN was modeled with p outputs, representing prediction steps. The last output was used to generate the fitness for the GNMRAC. In order to correctly model the ROP, several architectures were analyzed and the one with best results was a dynamic neural network using a time factor to simulate a short-term memory.

Neural networks may represent the concept of memory by two means. By itself, the NN structure characterizes a “long-term memory” device, as it represents the data input-output relationship in the net’s synaptic weights. “Long-term memory” is associated with the fact that, once the net is trained, it keeps the weights-and therefore the information-as long as it is required by the user. Besides the long-term memory, dynamic neural networks present a second type of memory, “short-term memory.” It temporarily stores information needed for the training and testing iterations, usually bringing data from one step to the next. Short-term memory structures can be implemented through the use of time delays at the input layer of the network.

A tapped delay line memory is the simplest and most common form of short-term memory. For instance, an input signal denoted by a discrete-time sequence {x(n)} would have a response given by x(n-1) for each delay unit.

The neural network used in the problem described herein may be classified as a dynamic network, as it uses the concept of short-term memory. The ARX model structure used is shown in Fig. 2. The architecture used is a feed-forward network, trained with the Levenberg-Marquardt algorithm.⁶

The three external inputs for the neural network are given by u₁(t) = TVD(t), for true vertical depth; u₂(t) = N(t), for the rotation of the drillstring; and by u₃(t) = WOB(t), for the weight on bit. Also, two past values of the ROP are given as inputs, represented by y₁(t) = ROP(t-1) and y₁(t-2) = ROP(t-2). The outputs of the neural net are the predicted penetration rate from the current up to the p-th step, represented by . The input vector, or regressor vector, for the problem proposed herein is then given by Eq. 1.

The data used to train and test the network was collected from the mud-logging service of an offshore field located in the Campos Basin in Brazil. It refers to seven wells drilled with equal-diameter bits. As the wells are in the same field and therefore near each other, a similar lithology is expected. The role of the depth parameter introduced as an input is to represent this lithological similarity among the wells. The methodology was applied taking each of the seven wells as the training set and compounding the testing set with the other six wells. The next section presents a few aspects related to genetic algorithms, which also compose the system presented.⁷

GENETIC NEURO-MODEL REFERENCE ADAPTIVE CONTROLLER (GNMRAC)

One of the main characteristics of an Adaptive Controller (AC) is its capacity of adjusting the controller sets during the control operations. The adjustment mechanism is usually an important component of an AC and might be seen as the source of the AC robustness. There are two major types of ACs: the self-tuning regulator and the Model Reference Adaptive Controller (MRAC),⁸ whose architecture was adopted in the present work.

Genetic algorithms are search algorithms based on natural selection mechanisms and Darwin’s theory of evolution. Traditionally, GAs have been applied to solve optimization problems. Some particularities of the GA adopted herein will be discussed in this section. Goldberg presents more detailed information about the general mechanisms of a basic genetic algorithm.⁹

The GNMRAC shown in Fig. 3 was applied as a control tool in order to find the parameters that should be set if a desired ROP is to be used. As mentioned earlier, controlling the ROP is usually limited to special drilling situations, such as formation logging tool runs. Therefore, the GNMRAC may be activated at any time during the drilling operation, depending on the operation being carried out. Basically, given a determined depth, the GA should find the WOB and the rotation that will generate the desired ROP, adjusted as the set point. When activated, it acts in real time, changing the settings of an actuator that provides the parameter’s values for the plant. In Fig. 3, N’ and WOB’ represent the parametric values analyzed by the GNMRAC, while N and WOB represent the setting values shown to the actuators. The activation was simulated choosing a random point in a data set to simulate, for instance, the moment when a logging tool would be run in the well.

The chromosome, or the population of abstract representation, should then represent the two controllable drilling variables, the rotation (N) and WOB. The number of genes for each variable was set to be ngenes = 10, leading to a chromosome of twenty genes: the first half representing the WOB and the second, the rotation. The data structure adopted is a string of binary digits. The size of the population was set to be sizepop = 30 individuals and the number of generations, to niterations = 100.

The decimal-converted numbers represented by each half of the chromosome, a₁ and a₂, are in the real interval [0, 2^ngenes-1], and in order to convert them into the range for each variable, finding WOB’ (x1) and N’ (x2) represented by the chromosome, Equations 2 and 3 were used.

The variables maxwob, minwob, maxrpm and minrpm represent, respectively, the maximum and minimum value of the WOB and of the rotation taken from the entire seven-wells data set, and are equal to 99.8 and 0 klb, and 227 and 24 rpm, respectively.

The connection between the controllable drilling parameters represented by the chromosome and the ROP that they generate is given by the neural network. For each population during the GNMRAC process, the NN was used to find a fitness for the individuals. The variables x₁ and x₂ defined in Eq. 2 and 3, along with the pre-established depth, are used as inputs to the adjust mechanism block, which then generates a 10-step prediction for the ROP.

Then the GA is able to order the population according to the proximity of the NN output, the ROP_i and a set point, for instance setpoint = 15. The goal of the GNMRAC is to find individuals with ROPs as close to the set point as possible. Therefore, the lesser the difference of ROP_i-setpoint (error measure), the better. The absolute fitness, AbsFitness, was then computed as the inverse of the absolute difference, as shown in Eq. 4.

To find the normalized fitness for each individual, the absolute fitnesses of all individuals were summed as the ftotal, which divided the absolute fitness of each individual to find the fitness, as shown in Eq. 5.

The fitness of each individual was normalized to simulate a roulette wheel with slots sized according to them, so it is essential to compute a cumulative fitness. To choose a parent, a random number between 0 and 1 is generated and the individual with the next greater cumulative fitness is picked to procreate. Table 1 shows an example of an ordered population with the fitness and cumulative fitness computed. The chromosomes of the individuals, as well as the WOB and rotation values, are also shown.

TABLE 1. Ordered GA population

In the GNMRAC applied in the methodology, crossing over and mutation were the genetic operators adopted. The chances for each operator were set to bet respectively 0.95 and 0.05. The controller operational mode might be represented by the genetic algorithm represented in Fig. 4. Two stopping criteria were used: maximum number of generations (niterations) and similarity between the first and last individuals of the ordered population (if they had the same chromosome, then stop).

In order to apply the NN, and consequently the GA, previous values of the ROP are needed, as well as a depth. A point between 1/3 and 2/3 of the size of the data set from one of the test wells was randomly picked. It was chosen to be the point where the GA was activated, simulating a startup in the control system at a random drilling depth.

APPLICATION RESULTS

As mentioned, the results for the system identification problem were obtained with an ARX Neural Network, using a feed-forward architecture with sigmoid neurons and trained with the Levenberg-Marquardt algorithm.

Field data has a great noise associated, and in order to intensify the data trends and reduce its randomness, smoothing was applied to each well data. This technique consists of representing a data group through a central value, which might be the average or the median. It is accomplished by the method of moving averages (or running averages) in which the Y-value of each data point is averaged with that of some of its neighbors. These values are then connected, constructing a smooth line, possibly curved.¹⁰ This process might be repeated a number of times before an acceptably smooth curve is achieved.

For the wells composing the data set, 30-iteration smoothing, using the average of five points as a central value, was applied. It is also important to mention that outliers were removed from the data set.

After the outliers removal and smoothing, the prediction step was set for p = 10 and the data from the seven wells were used to train the NN, one at a time. After the training, the network was presented to six different testing sets, corresponding to the six other wells. The results were analyzed with the Pearson correlation factor, and ranged from 0.888 to 0.988 for the test wells. Figure 5 shows the results for one of these wells, where a correlation of 0.988 was found.

Once acceptable correlations were found for the ROP modeling problem, the NN was used during a GA process to define the parameters that should be used to generate a pre-set ROP, as previously mentioned. The methodology was applied to the test wells composing the total data set. The neural network trained during the system identification phase was used to simulate the plant behavior after the control system was activated. A well was randomly picked in order to simulate the activation of the ROP control system. A random point ranging from 1/3 to 2/3 of the well data set size was picked and set to be the startup point. The interval between this point and the end of the well data set was then divided in three to simulate two setpoint adjustments. It supposed, for instance, that three logging tools were to be used during the drilling operation, each requiring different ROPs. Figure 6 shows the results for one of the simulations, using 15, 10 and 25 m/hr, respectively, as set points.

The system output correctly followed the adjusted set points, as shown in Fig. 6. It is also explicit that the system presented a short rise time and has shown neither oscillation nor overshot. Similar results were found for several other simulations. The results highlight the GNMRAC automatic adaptation efficiency to provide a trustful ROP control in several operational conditions.

CONCLUSIONS

The Genetic Neuro-Model Reference Adaptive Controller (GNMRAC) was introduced. The application of the “auto-regressive with extra inputs” neural network to oil field data has generated a successful model of the plant. The correlations achieved with the methodology ranged from 0.888 to 0.988 for the testing sets, which may be considered an overwhelming result in the petroleum industry, where most of the operations involving rock-equipment interactions are associated with uncertainty and imprecision.

Several authors have proposed mathematical approaches to model the ROP. Neural network-based models show some difficulty in correctly characterizing wells not seen by the net during the training phase. It is important to mention that real field data usually is associated with acquisition problems, such as the presence of outliers and noise. Data preparation is therefore essential, especially if the methodology is based on information extracted from field data.

In addition, the reference model NN architecture provides to the GNMRAC the capacity of controlling the ROP with a p-step advance. This characteristic enables the controller to take predictive actions to correct the ROP.

Furthermore, the main difficulty is probably associated with the complexity of the relationships between the ROP and other drilling parameters. These relationships are possibly related to local features, especially involving rock-formation interaction. The use of a short-term memory in the neural network architecture has shown to be a solution to the ROP modeling problem. When ROP information from the well is used as an input to the network-even if it was not trained with that information-a good prediction is achieved by the ARX NN.

Once the NN is able to model the ROP, input-parameter relationships might be analyzed. The NN output is then used as the basis for the control system. Real-time logging tools often require the driller to slow drilling to a certain ROP, and therefore a reliable control system is essential.

Neuro-genetics algorithms have proven their applicability as a robust adaptive control tool. It is a global search technique that does not need to assume that the search space is neither continuous nor differentiable. Genetic algorithms were successfully tested in simulations, converging to the setpoint in every run. Although the GNMRAC was not tested in a physical apparatus, the high quality of the results indicates that it can be efficiently applied to well drilling field operations.  

ACKNOWLEDGEMENT

This article was prepared from “Genetic Neuro-Model Reference Adaptive Controller for Petroleum Wells Drilling Operations” presented at the International Conference on Computational Intelligence for Modeling Control and Automation, Sydney, Australia, 2006. © [2006] IEEE. Reprinted, with permission, from IEEE Computer Society - Washington, DC, USA.

LITERATURE CITED

 ¹ R.V. Barragan, “Otimização dos parâmetros mecânicos nas brocas para obter o custo mínimo de uma fase de um poço,” (In Portuguese) master’s dissertation, Faculdade de Engenharia Mecânica, Universidade Estadual de Campinas. Campinas, Brazil, 1995.
² A.T. Bourgoyne Jr., F.S. Young Jr., “A multiple regression approach to optimal drilling and abnormal detection,” Society of Petroleum Engineers Journal, 14(4), 371-384, 1974.
³ H.I. Bilgesu, L.T. Tetrick, U. Altmis, S. Mohaghegh, S. Ameri, “A new approach for the prediction of rate of penetration values,” Presented at SPE Eastern Regional Meeting, Lexington, USA, 1997.
⁴ M. Norgaard, O. Ravn, N.K. Poulsen, L.K. Hansen, Neural Networks for Modeling and Control of Dynamic Systems: A Practitioner’s Handbook, Springer, 2000.
⁵ S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice-Hall, 1999.
⁶ D. Marquardt, “An algorithm for least-squares estimations of nonlinear parameters,” SIAM Journal on Applied Mathematics, Vol. 11(2), 431-441, 1963.
⁷ T. C. Fonseca, J. R. P. Mendes, A. B. S. Serapião, and I. R. Guilherme, “Application of ARX Neural Networks to Model the Rate of Penetration of Petroleum Wells Drilling”, in Proc. The IASTED International Conference on Computational Intelligence, San Francisco, USA, 2006, 154-159.
⁸ K.J. Astrom, B. Wittenmark, Adaptive Control, Addison-Wesley, 1989.
⁹ D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989.
¹⁰ A.F. Siegel, Statistics and Data Analysis, John Wiley & Sons, 1988.