Jekyll2021-02-26T03:45:27-06:00https://www.anandk.dev/feed.xmlAnand’s technical musingsOver 20 years of experience designing and developing applications on IBM i (AS/400, iSeries) system. Hands-on knowledge of COBOL, RPG36, ILE RPG, RPG FREE format, CLLE, SQL, etc. Excellent knowledge of JavaScript, NodeJS, Python. Instrumental in writing web-services/RESTful APIs. Application modernization, application maintenance on IBM i (AS400, iSeries)Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devMachine learning/Artificial Intelligence on IBM i2021-02-21T00:00:00-06:002021-02-21T00:00:00-06:00https://www.anandk.dev/2021/02/Machine-learning-on-ibmi<p style="text-align: justify;"><img src="https://www.anandk.dev/assets/images/forposts/ibmi-ml/ibmi-ml-header.png" alt="ibmi-ml-header" class="align-center" />
In this article, let’s see a simple machine learning model implemented on IBM i using Python that can predict future values by learning from current data in the system.</p>
<p style="text-align: justify;">As per this article by <a href="https://www.gartner.com/smarterwithgartner/gartner-top-10-trends-in-data-and-analytics-for-2020/">Gartner</a>, “Data and analytics combined with artificial intelligence (AI) technologies will be paramount in the effort to predict, prepare and respond in a proactive and accelerated manner to a global crisis and its aftermath.” The report goes on to say, “By the end of 2024, 75% of enterprises will shift from piloting to operationalizing AI, driving a 5X increase in streaming data and analytics infrastructures.”</p>
<p style="text-align: justify;">Hence, it is essential to see how we can ready ourselves in the IBM i world to start using these technologies. There are many applications running on the IBM i having a huge amount of data that can be leveraged to get some foresight.</p>
<p style="text-align: justify;">Please note, this article is a demonstration of and not a how-to guide of implementing and running a machine learning algorithm on the IBM i due to the vastness of the subject. For those who are interested to study the subject in detail, I have given links at the end of the article on where and how to start.</p>
<p style="text-align: justify;">I have assumed basic Python programming knowledge. Please refer to my <a href="https://www.anandk.dev/2020/12/Create-pdf-python-ibmi.html">earlier article</a> which demonstrates the use of Python on the IBM i.</p>
<blockquote>
<p>Note: The code and data for this article is available to download <a href="https://github.com/anand-khekale/ibmi-ml/archive/master.zip">here</a>.</p>
</blockquote>
<h2 id="what-is-machine-learning">What is Machine Learning?</h2>
<p style="text-align: justify;">As per <a href="https://developers.google.com/machine-learning/glossary#m">Google</a>, Machine learning is, “A program or system that builds (trains) a predictive model from input data. The system uses the learned model to make useful predictions from new (never-before-seen) data drawn from the same distribution as the one used to train the model. Machine learning also refers to the field of study concerned with these programs or systems.”</p>
<p style="text-align: justify;">The main difference between traditional programming versus machine learning is as below. We are going to see it in action.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-ml/classical-vs-ml-pgm.png" alt="What-is-ml" class="align-center" /></p>
<h2 id="python-libraries-we-will-be-using">Python libraries we will be using</h2>
<p style="text-align: justify;">We will be using the below Python libraries for this task. All the below libraries are already installed on PUB400 globally hence we will not be creating any virtual environment. Please install those if you are trying them on a different server, each library’s documentation provides ways to install those. These are all open-source and free.</p>
<p>1. <a href="https://numpy.org/">NumPy</a></p>
<p style="text-align: justify;">NumPy is a library to handle large, multi-dimensional arrays and matrices. It also supports high-level mathematical functions to operate on these arrays.</p>
<p>2. <a href="https://pandas.pydata.org/">Pandas</a></p>
<p style="text-align: justify;">Pandas is a library providing high-performance, easy-to-use data structures and data analysis tools. It offers data structures and operations for manipulating numerical tables and time series.</p>
<p>3. <a href="https://scikit-learn.org/stable/">Scikit-learn</a></p>
<p style="text-align: justify;">Scikit-learn <a href="https://scikit-learn.org/stable/"></a>provides a range of supervised and unsupervised machine learning algorithms with a standard interface in Python.</p>
<p>4. <a href="https://docs.python.org/3/library/pickle.html">Pickle</a></p>
<p style="text-align: justify;">Pickle is used to save a Python object to a file, for example, a machine learning model which we can load later to make predictions as and when needed.</p>
<h2 id="some-machine-learning-terms">Some Machine learning terms:</h2>
<p style="text-align: justify;">I am giving here a high-level definitions of the terms often used in machine learning so that those will be easy to understand while we build the programs on the system. Please refer to <a href="https://developers.google.com/machine-learning/glossary#a">Google’s</a> glossary for more such terms.</p>
<p>1. Features</p>
<p style="text-align: justify;"><a href="https://developers.google.com/machine-learning/glossary#feature">Features</a> (represented as X) are the parameters using which we will be predicting a value (represented as y). e.g. For predicting the price of a house (y), we will be using parameters such as locality, number of bedrooms, area, etc.</p>
<p>2. Labels</p>
<p style="text-align: justify;"><a href="https://developers.google.com/machine-learning/glossary#label">Labels</a> are the values (y) we give to the machine learning model to learn the relation between feature and label (X & y). Labels are the actual price of houses in a dataset.</p>
<p>3. Model</p>
<p style="text-align: justify;">A machine learning <a href="https://developers.google.com/machine-learning/glossary#model">model</a> is a trained instance of an algorithm. Training here means the input data (Features & Labels) is used to recognize patterns in it. Using this model we can predict patterns/predictions on a new set of data.</p>
<p>4. Supervised and Unsupervised learning</p>
<p style="text-align: justify;"><a href="https://developers.google.com/machine-learning/glossary#supervised-machine-learning">Supervised learning</a> means when the model is provided with both features and labels (X & y), that is, we provide some pattern of data to the model. In house price prediction example, the number of bedrooms (X) and price of the house(y).</p>
<p style="text-align: justify;"><a href="https://developers.google.com/machine-learning/glossary#unsupervised-machine-learning">Unsupervised learning</a> means there is no label and the machine learning algorithm has to identify the pattern in it. e.g. grouping of customers based on their spending, earning power, etc.</p>
<p>5. Training</p>
<p style="text-align: justify;"><a href="https://developers.google.com/machine-learning/glossary#training">Training</a> is a process by which ideal parameters for a model are determined.</p>
<h2 id="our-machine-learning-problem">Our machine learning problem</h2>
<p style="text-align: justify;">To keep things simple to understand, we will take a simple dataset that contains Years of Experience as X and Salary as y. Using this dataset, we will build the model to predict salary when we input Years of experience into the model.</p>
<p style="text-align: justify;">Similarly, simple to very complex models can be built on IBM i, few examples that come to my mind are:</p>
<ol>
<li>Future raw material requirement.</li>
<li>Analyzing quality of material from their images using <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">Convolution Neural Network</a>.</li>
<li>Future Revenue prediction based on the past data.</li>
</ol>
<p>Likewise, the possibilities are endless.</p>
<h2 id="implementation-and-testing-of-the-algorithm">Implementation and testing of the algorithm</h2>
<h3 id="dataset">Dataset:</h3>
<p style="text-align: justify;">We will use a dataset from <a href="https://www.kaggle.com/">Kaggle</a> which hosts many data science and machine learning datasets, code, and environments. In a typical environment on IBM i, we will be using data from the system itself, but since this is just a demonstration so any data should give an idea about the concept.</p>
<p>Our data is a file named ‘Salary_Data.csv’ which contains just 31 records including a header row.</p>
<p>Below are just few rows from our dataset.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-ml/salary-dataset.png" alt="salary-dataset" class="align-center" /></p>
<h3 id="linear-regression">Linear Regression:</h3>
<p style="text-align: justify;">I just want to give a small introduction to the algorithm. When we are working with only one input variable, it is termed Simple Linear Regression. It attempts to find the relationship between two variables using a linear equation.</p>
<p style="text-align: justify;">The linear equation is <code class="language-plaintext highlighter-rouge">y=mx + b</code>. Here we are predicting <code class="language-plaintext highlighter-rouge">y</code> by fitting the <code class="language-plaintext highlighter-rouge">m</code>, which is the slope of the line, and <code class="language-plaintext highlighter-rouge">b</code>, which is the y-intercept. Here we have <code class="language-plaintext highlighter-rouge">x</code> - years of experience as input.</p>
<p>The graph of which can be plotted as below.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-ml/linear-regression.png" alt="Linear-regression" class="align-center" /></p>
<p style="text-align: justify;">The algorithm tries to find out <code class="language-plaintext highlighter-rouge">m</code> and <code class="language-plaintext highlighter-rouge">b</code> by minimizing the error when compared to values given in our labels <code class="language-plaintext highlighter-rouge">y</code>. An algorithm named <a href="https://developers.google.com/machine-learning/glossary#gradient-descent">Gradient Descent</a> is used to arrive at the values of <code class="language-plaintext highlighter-rouge">m</code> and <code class="language-plaintext highlighter-rouge">b</code>.</p>
<h3 id="program">Program:</h3>
<p>Let’s see the implementation of the program.</p>
<p style="text-align: justify;">First, we will write a program that will take the data, assign it to <code class="language-plaintext highlighter-rouge">X</code> and <code class="language-plaintext highlighter-rouge">y</code> variable, split it into the Train and Test set, and then it is used to train the model.</p>
<p style="text-align: justify;">The code for <code class="language-plaintext highlighter-rouge">trainmodel.py</code> is as below. Please go through the comments to understand each line.</p>
<p style="text-align: justify;">Change your current directory to <code class="language-plaintext highlighter-rouge">ibmi-ml</code> and call the program using the command <code class="language-plaintext highlighter-rouge">python3 trainmodel.py</code>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cd </span>ibmi-ml
<span class="nv">$ </span>python3 trainmodel.py
</code></pre></div></div>
<p>You should get the below messages after successful run.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>python3 trainmodel.py
Linear Regression model built with score: 0.988169515729126
Model saved <span class="k">in </span>file finalized_model.sav
</code></pre></div></div>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># NumPy is used to handle the data to put it in a 2-dimensional array.
</span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="n">pd</span> <span class="c1"># Pandas is used to read the data from database or csv.
# Import the LinearRegression class of Sci-kit learn library.
</span><span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LinearRegression</span>
<span class="c1"># Import r2_score to measure the performance of our model.
</span><span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">r2_score</span>
<span class="c1"># Split the data as train and test set to test the model.
</span><span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
<span class="kn">import</span> <span class="nn">pickle</span> <span class="c1"># To save the trained model into a file.
</span>
<span class="c1"># df is a short form of the dataframe, we read the data from the csv and put it in a pandas dataframe object.
</span><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'Salary_Data.csv'</span><span class="p">)</span>
<span class="c1"># Reshape the data, meaning take one column, the feature, and put it in a 2-D array in a variable named X.
</span><span class="n">X</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s">'YearsExperience'</span><span class="p">].</span><span class="n">values</span><span class="p">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="c1"># Reshape the data,same for the values we are giving to the model based on which it will predict future values in
# variable y.
</span><span class="n">y</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s">'Salary'</span><span class="p">].</span><span class="n">values</span><span class="p">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="c1"># Split the data into the Train and Test sections so that we can measure the performance of the model.
</span><span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">test_size</span><span class="o">=</span><span class="mf">0.2</span><span class="p">,</span> <span class="n">random_state</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="c1"># Initialize the Scikit learn LinearRegression Class and fit (train the model with training data)
</span><span class="n">reg</span> <span class="o">=</span> <span class="n">LinearRegression</span><span class="p">()</span>
<span class="n">reg</span><span class="p">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
<span class="c1"># Use Test data to derive the prediction in y_pred array
</span><span class="n">y_pred</span> <span class="o">=</span> <span class="n">reg</span><span class="p">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="c1"># Print the score of the model by comparing it with predicted value and actual test values.
# Score closer to 1 is better performance.
</span><span class="k">print</span><span class="p">(</span><span class="s">f'Linear Regression model built with score: </span><span class="si">{</span><span class="n">r2_score</span><span class="p">(</span><span class="n">y_test</span><span class="p">,</span> <span class="n">y_pred</span><span class="p">)</span><span class="si">}</span><span class="s">'</span><span class="p">)</span>
<span class="c1"># Save the model to the disk for future use.
</span><span class="n">filename</span> <span class="o">=</span> <span class="s">'finalized_model.sav'</span>
<span class="n">pickle</span><span class="p">.</span><span class="n">dump</span><span class="p">(</span><span class="n">reg</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="s">'Model saved in file'</span><span class="p">,</span> <span class="n">filename</span><span class="p">)</span>
</code></pre></div></div>
<p style="text-align: justify;">To demonstrate data handling by NumPy, here is the content of variable <code class="language-plaintext highlighter-rouge">X</code> using Python command <code class="language-plaintext highlighter-rouge">print(X)</code>. Variable <code class="language-plaintext highlighter-rouge">X</code> contains “Years of experience”.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[[</span> <span class="mf">1.1</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">1.3</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">1.5</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">2.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">2.2</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">2.9</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">3.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">3.2</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">3.2</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">3.7</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">3.9</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">4.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">4.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">4.1</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">4.5</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">4.9</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">5.1</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">5.3</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">5.9</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">6.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">6.8</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">7.1</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">7.9</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">8.2</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">8.7</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">9.</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">9.5</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">9.6</span><span class="p">]</span>
<span class="p">[</span><span class="mf">10.3</span><span class="p">]</span>
<span class="p">[</span><span class="mf">10.5</span><span class="p">]]</span>
</code></pre></div></div>
<p style="text-align: justify;">Similarly, our corresponding <code class="language-plaintext highlighter-rouge">y</code> variable, “Salary” looks like the below when saved in a 2-dimensional array.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[[</span> <span class="mf">39343.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">46205.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">37731.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">43525.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">39891.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">56642.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">60150.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">54445.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">64445.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">57189.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">63218.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">55794.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">56957.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">57081.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">61111.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">67938.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">66029.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">83088.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">81363.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">93940.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">91738.</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">98273.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">101302.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">113812.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">109431.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">105582.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">116969.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">112635.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">122391.</span><span class="p">]</span>
<span class="p">[</span><span class="mf">121872.</span><span class="p">]]</span>
</code></pre></div></div>
<p style="text-align: justify;">These arrays after splitting into Train and Test sets are passed to the LinearRegression class to its <code class="language-plaintext highlighter-rouge">fit</code> method. This method takes the data and finds out the co-relation between it, i.e. the values for <code class="language-plaintext highlighter-rouge">m</code> and <code class="language-plaintext highlighter-rouge">b</code>.</p>
<h3 id="how-to-re-use-the-model">How to re-use the model</h3>
<p style="text-align: justify;">We should be able to re-use the model on a new set of data as and when the business needs it. Hence in the above step, we stored the model in a file that can be used often as shown in the program <code class="language-plaintext highlighter-rouge">usemodel.py</code>.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pickle</span> <span class="c1"># Pickle is used to load the stored state of a python object.
</span>
<span class="c1"># Pass new value to predict from the loaded model.
</span><span class="n">exp</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="nb">input</span><span class="p">(</span><span class="s">'Enter years of experience: '</span><span class="p">))</span>
<span class="n">filename</span> <span class="o">=</span> <span class="s">'finalized_model.sav'</span>
<span class="c1"># Load the model from the disk
</span><span class="n">loaded_model</span> <span class="o">=</span> <span class="n">pickle</span><span class="p">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s">'rb'</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">loaded_model</span><span class="p">.</span><span class="n">predict</span><span class="p">([[</span><span class="n">exp</span><span class="p">]])</span>
<span class="k">print</span><span class="p">(</span><span class="s">'The new predicted salary is'</span><span class="p">,</span> <span class="n">result</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span>
</code></pre></div></div>
<p>Our re-usable machine learning model is ready to serve the users.
Call the program using the command <code class="language-plaintext highlighter-rouge">python3 usemodel.py</code>.</p>
<h3 id="limitations">Limitations:</h3>
<p style="text-align: justify;">Since this is a very simple model using just one input variable, the predictions will not match reality beyond a certain limit. For example, if we input experience as 40 years, it will predict the salary as 399283.10, which may not be true. A more complex model needs to be configured for accuracy with more features such as education, domain knowledge, etc. Also, the model needs to be tuned with additional parameters such as <a href="https://en.wikipedia.org/wiki/Regularization_(mathematics)">regularization</a>, which is an advanced topic.</p>
<p style="text-align: justify;">To keep the scope of the article very basic and simple to understand, those additional techniques are omitted from the article.</p>
<h2 id="how-to-install-and-run-the-programs">How to install and run the programs</h2>
<ol>
<li>Download the utility folder from <a href="https://github.com/anand-khekale/ibmi-ml/archive/master.zip">here</a>.</li>
<li>In the IFS create directory <code class="language-plaintext highlighter-rouge">mkdir ibmi-ml</code>.</li>
<li>Upload the content of the folder in IFS in directory <code class="language-plaintext highlighter-rouge">ibmi-ml</code>.</li>
<li>Login to SSH or CALL QP2TERM session.</li>
<li>Change the current directory to <code class="language-plaintext highlighter-rouge">ibmi-ml</code> using <code class="language-plaintext highlighter-rouge">cd ibmi-ml</code>.</li>
<li>Create the model using command <code class="language-plaintext highlighter-rouge">python3 trainmodel.py</code>. This is a one-time step.</li>
<li>Test the model by inputting various experience levels using <code class="language-plaintext highlighter-rouge">python3 usemodel.py</code></li>
</ol>
<h2 id="useful-links-to-learn-machine-learning">Useful links to learn machine learning</h2>
<p style="text-align: justify;">These are some of the links I found useful while learning about these topics. This is not an exhaustive list but good enough to start with.</p>
<h3 id="courses">Courses:</h3>
<ol>
<li>Machine Learning by Andrew Ng on <a href="https://coursera.org/share/847634fa5adb107f23f1e7cdf4d8ba0c">Coursera</a></li>
<li>Carnegie Mellon University Lectures - <a href="http://www.cs.cmu.edu/~ninamf/courses/601sp15/lectures.shtml">Link</a></li>
<li>Machine Learning Crash Course by Google - <a href="https://developers.google.com/machine-learning/crash-course">Link</a></li>
</ol>
<h3 id="books">Books:</h3>
<ol>
<li>Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow- <a href="https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1492032646/ref=sr_1_1?crid=2FRGSEF8L0OXE&dchild=1&keywords=hands-on+machine+learning+with+scikit-learn%2C+keras%2C+and+tensorflow&qid=1614155537&sprefix=hands-on+%2Caps%2C360&sr=8-1">Link</a></li>
<li>Python for Data Analysis - <a href="https://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1491957662/ref=sr_1_1?dchild=1&keywords=Python+for+Data+Analysis&qid=1614155656&sr=8-1">Link</a></li>
<li>The Hundred-Page Machine Learning Book - <a href="https://www.amazon.com/Hundred-Page-Machine-Learning-Book/dp/199957950X/ref=sr_1_1?dchild=1&keywords=The+Hundred-Page+Machine+Learning+Book&qid=1614155725&sr=8-1">Link</a></li>
</ol>
<h2 id="special-thanks">Special Thanks</h2>
<ol>
<li><a href="http://pub400.COM">PUB400.COM</a> - Such a fantastic server to try out new things on IBM i.</li>
<li><a href="http://papaya.io">Papaya.io</a> - Web-based image editing tool, very easy and fast for image editing.</li>
</ol>
<h2 id="conclusion">Conclusion</h2>
<p style="text-align: justify;">Machine learning and Artificial Intelligence are becoming a go-to field for data-analytics, robotics, decision-making in every domain. IBM i has a huge amount of data to its advantage as the applications are running for so many years for Banking, Manufacturing, Supply Chain, and numerous other domains. The availability of an environment (PASE for i) which supports open-source on the system and the vast open-source free-to-use machine learning libraries is a win-win situation for any CIO. These can be leveraged for the better good of business and industry.</p>
<p style="text-align: justify;">As always, request you to get in touch with me on <a href="https://www.linkedin.com/in/anandkdev">LinkedIn</a>/<a href="https://twitter.com/anandkdev">Twitter</a> or <a href="mailto:anand.khekale@gmail.com">email</a> with your valuable feedback. Thanks for reading!</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn this article, let’s see a simple machine learning model implemented on IBM i using Python that can predict future values by learning from current data in the system.How to generate PDFs using Python on IBM i2020-12-20T00:00:00-06:002020-12-20T00:00:00-06:00https://www.anandk.dev/2020/12/Create-pdf-python-ibmi<p style="text-align: justify;"><img src="https://www.anandk.dev/assets/images/forposts/ibmi-pdf/ibmi-pdf-header.png" alt="ibmi-pdf-header" class="align-center" />
In this article, let’s see how we can use Python on the IBM i to generate well-formatted PDFs. The advantage of using Python for this is, it provides various packages so that the formatting, placement of the tables, etc. are handled easily; we will be using one of such packages.</p>
<p style="text-align: justify;">I have assumed a basic knowledge of Python, but have given enough details for the beginners.
Let’s see how we can start using Python on the IBM i.</p>
<blockquote>
<p>Note: The code for this utility can be downloaded from <a href="https://github.com/anand-khekale/ibmi-pdf/archive/main.zip">here</a>.</p>
</blockquote>
<h3 id="check-python-on-ibm-i">Check Python on IBM i</h3>
<p style="text-align: justify;">Check if your IBM i has Python installed in the open-source software by using the below command in CALL QP2TERM session (or SSH session). We will check the version of Python as well.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>which python3
/QOpenSys/pkgs/bin/python3
<span class="nv">$ </span>python3 <span class="nt">--version</span>
Python 3.6.12
</code></pre></div></div>
<p>Once we are sure we have Python 3 available on the system, we are all set to work on this utility.</p>
<h3 id="create-a-working-directory">Create a working Directory</h3>
<p>For this utility, let’s start with creating a directory and working from that directory.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">mkdir </span>ibmi-pdf
<span class="nv">$ </span><span class="nb">cd </span>ibmi-pdf
</code></pre></div></div>
<h3 id="python-virtual-environments">Python Virtual Environments</h3>
<p style="text-align: justify;">Before we start installing the necessary libraries needed for our task, let’s first introduce ourselves to the virtual environment in Python.</p>
<p style="text-align: justify;">Python comes with standard packages. It also provides third-party packages that we need to install before we can use those. By creating a virtual environment, we are scoping the use of those third-party libraries just for our use-case without affecting the standard packages and their versions. It gives more advantage to work independently and help keep the package version intact just for our case.</p>
<p style="text-align: justify;">Many times on the IBM i, we may not have permission to install the package globally, so it is a good practice to create a virtual environment and install the package in it.
A virtual environment is a self-contained directory tree that contains a Python installation for a particular version of Python, plus several additional packages.</p>
<h3 id="how-to-create-virtual-environment">How to create Virtual Environment</h3>
<p>A virtual environment can be created as shown below; it will create one named <code class="language-plaintext highlighter-rouge">venv1</code> . (You may provide any name to the environment.)</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>python3 <span class="nt">-m</span> venv <span class="nt">--system-site-packages</span> venv1
</code></pre></div></div>
<p>After the completion of the above command, a folder by the name <code class="language-plaintext highlighter-rouge">venv1</code> will be created in our directory <code class="language-plaintext highlighter-rouge">ibmi-pdf</code>.</p>
<p>Once the virtual environment is created, we have to activate it by using the below command.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">.</span> venv1/bin/activate
venv1 <span class="err">$</span>
</code></pre></div></div>
<p style="text-align: justify;">Our $ prompt will change with the name of the virtual environment after this step. It means that the virtual environment is now active.</p>
<h3 id="installing-the-packages">Installing the packages</h3>
<p>Now that our virtual environment is ready, it’s time to install a specific package for PDF generation, which is <a href="https://pypi.org/project/fpdf/">FPDF</a>.</p>
<p style="text-align: justify;">We will be installing the package using a Python package installer known as <code class="language-plaintext highlighter-rouge">pip</code>. This installer searches the package in the Python Package Index and downloads all the dependencies for us.
We will install it by issuing the below command.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>venv1 <span class="nv">$ </span>pip3 <span class="nb">install </span>fpdf
</code></pre></div></div>
<p>Now that the package is installed, let’s start working on the code to generate PDFs.</p>
<h3 id="fetch-data-from-db2-using-ibm_db_dbi">Fetch Data from DB2 using ibm_db_dbi</h3>
<p style="text-align: justify;">We will be fetching the data from DB2 using IBM-provided Python package <code class="language-plaintext highlighter-rouge">ibm_db_dbi</code> . This package is available by default, there is no need to install it. The below code demonstrates how it fetches the data. We are using the demo file provided by IBM, i.e. <code class="language-plaintext highlighter-rouge">QIWS/QCUSTCDT</code></p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">ibm_db_dbi</span> <span class="k">as</span> <span class="n">db2</span> <span class="c1">#Import the package
</span><span class="n">conn</span><span class="o">=</span> <span class="n">db2</span><span class="p">.</span><span class="n">connect</span><span class="p">()</span> <span class="c1"># Make a connection
</span><span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="p">.</span><span class="n">cursor</span><span class="p">()</span>
<span class="n">cur</span><span class="p">.</span><span class="n">execute</span><span class="p">(</span><span class="s">"SELECT CUSNUM, LSTNAM, BALDUE, CDTLMT FROM QIWS.QCUSTCDT"</span><span class="p">)</span>
<span class="c1"># CursorByName will provide data in Key:Value pair, which we will use while generating
# the PDF
</span><span class="k">class</span> <span class="nc">CursorByName</span><span class="p">():</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">cursor</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span> <span class="o">=</span> <span class="n">cursor</span>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span>
<span class="k">def</span> <span class="nf">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span><span class="p">.</span><span class="n">__next__</span><span class="p">()</span>
<span class="k">return</span> <span class="p">{</span><span class="n">description</span><span class="p">[</span><span class="mi">0</span><span class="p">]:</span> <span class="n">row</span><span class="p">[</span><span class="n">col</span><span class="p">]</span> <span class="k">for</span> <span class="n">col</span><span class="p">,</span> <span class="n">description</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span><span class="p">.</span><span class="n">description</span><span class="p">)}</span>
<span class="n">data</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">CursorByName</span><span class="p">(</span><span class="n">cur</span><span class="p">):</span>
<span class="n">data</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="c1">#Get data from the cursor and store it in an array
</span></code></pre></div></div>
<h3 id="add-pdf-processing">Add PDF Processing</h3>
<p style="text-align: justify;">Using the above code, we can fetch the data from the DB2 table. Now we will see how we can add it in a well-formatted table in a PDF document.
As discussed earlier, we will be importing the FPDF package for it. See the code below for PDF processing.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">fpdf</span> <span class="kn">import</span> <span class="n">FPDF</span>
<span class="n">pdf</span> <span class="o">=</span> <span class="n">FPDF</span><span class="p">()</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">add_page</span><span class="p">()</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_left_margin</span><span class="p">(</span><span class="n">margin</span><span class="o">=</span><span class="mi">40</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'B'</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_fill_color</span><span class="p">(</span><span class="mi">193</span><span class="p">,</span><span class="mi">229</span><span class="p">,</span><span class="mi">252</span><span class="p">)</span>
<span class="c1">#Set the table headers
</span><span class="n">fill_color</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">40</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Customer Number'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Last Name'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Balance Due'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Credit Limit'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="c1"># Style set up for rows
</span><span class="n">pdf</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_fill_color</span><span class="p">(</span><span class="mi">235</span><span class="p">,</span><span class="mi">236</span><span class="p">,</span><span class="mi">236</span><span class="p">)</span>
<span class="c1"># Process the database rows to print to the PDF
</span><span class="n">fill_color</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="n">cus_num</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'CUSNUM'</span><span class="p">])</span>
<span class="n">lst_nam</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="s">'LSTNAM'</span><span class="p">]</span>
<span class="n">bal_due</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'BALDUE'</span><span class="p">])</span>
<span class="n">cdt_lmt</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'CDTLMT'</span><span class="p">])</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">40</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">cus_num</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">lst_nam</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">bal_due</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'R'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">cdt_lmt</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'R'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="c1"># Flip the fill color for alternate row.
</span> <span class="k">if</span> <span class="p">(</span><span class="n">fill_color</span><span class="o">==</span><span class="mi">0</span><span class="p">):</span>
<span class="n">fill_color</span><span class="o">=</span><span class="mi">1</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">fill_color</span><span class="o">=</span><span class="mi">0</span>
<span class="c1"># Output the PDF file
</span><span class="n">pdf</span><span class="p">.</span><span class="n">output</span><span class="p">(</span><span class="s">'customer.pdf'</span><span class="p">,</span><span class="s">'F'</span><span class="p">)</span>
</code></pre></div></div>
<p>A<code class="language-plaintext highlighter-rouge">customer.pdf</code> file is generated as displayed below.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-pdf/first_pdf.png" alt="first-pdf" class="align-center" /></p>
<p style="text-align: justify;">Now add more formatting to the PDF; add header and footer. Add the following code to get the desired effect.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">## Add Header and Footer to the page
</span><span class="k">class</span> <span class="nc">PDF</span><span class="p">(</span><span class="n">FPDF</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">header</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Logo
</span> <span class="bp">self</span><span class="p">.</span><span class="n">image</span><span class="p">(</span><span class="s">'ibmi.png'</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">20</span><span class="p">)</span>
<span class="c1"># Arial bold 15
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'B'</span><span class="p">,</span> <span class="mi">15</span><span class="p">)</span>
<span class="c1"># Move to the right
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">80</span><span class="p">)</span>
<span class="c1"># Title
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">30</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="s">'Customer balance due report '</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s">'C'</span><span class="p">)</span>
<span class="c1"># Line break
</span> <span class="bp">self</span><span class="p">.</span><span class="n">ln</span><span class="p">(</span><span class="mi">40</span><span class="p">)</span>
<span class="c1"># Add dashed line
</span> <span class="bp">self</span><span class="p">.</span><span class="n">dashed_line</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">200</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="c1"># Add line break after the dashed line
</span> <span class="bp">self</span><span class="p">.</span><span class="n">ln</span><span class="p">(</span><span class="mi">20</span><span class="p">)</span>
<span class="c1"># Page footer
</span> <span class="k">def</span> <span class="nf">footer</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Position at 1.5 cm from bottom
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_y</span><span class="p">(</span><span class="o">-</span><span class="mi">15</span><span class="p">)</span>
<span class="c1"># Arial italic 8
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'I'</span><span class="p">,</span> <span class="mi">8</span><span class="p">)</span>
<span class="c1"># Page number
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="s">'Page '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">page_no</span><span class="p">()),</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s">'C'</span><span class="p">)</span>
</code></pre></div></div>
<p style="text-align: justify;">With this, our PDF generation is complete. We get the final file as below, with the company logo, title, and footer. We can customize it any way we want. For simplicity, I am stopping here.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-pdf/final_pdf.png" alt="final-pdf" class="align-center" /></p>
<p>The final code for the utility is as below.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">fpdf</span> <span class="kn">import</span> <span class="n">FPDF</span>
<span class="kn">import</span> <span class="nn">ibm_db_dbi</span> <span class="k">as</span> <span class="n">db2</span> <span class="c1">#Import the package
</span>
<span class="c1"># Fetch data from the database
</span><span class="n">conn</span><span class="o">=</span> <span class="n">db2</span><span class="p">.</span><span class="n">connect</span><span class="p">()</span> <span class="c1"># Make a connection
</span><span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="p">.</span><span class="n">cursor</span><span class="p">()</span>
<span class="n">cur</span><span class="p">.</span><span class="n">execute</span><span class="p">(</span><span class="s">"SELECT CUSNUM, LSTNAM, BALDUE, CDTLMT FROM QIWS.QCUSTCDT"</span><span class="p">)</span>
<span class="c1"># CursorByName will provide data in Key:Value pair, which we will use while generating
# the PDF
</span><span class="k">class</span> <span class="nc">CursorByName</span><span class="p">():</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">cursor</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span> <span class="o">=</span> <span class="n">cursor</span>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span>
<span class="k">def</span> <span class="nf">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span><span class="p">.</span><span class="n">__next__</span><span class="p">()</span>
<span class="k">return</span> <span class="p">{</span><span class="n">description</span><span class="p">[</span><span class="mi">0</span><span class="p">]:</span> <span class="n">row</span><span class="p">[</span><span class="n">col</span><span class="p">]</span> <span class="k">for</span> <span class="n">col</span><span class="p">,</span> <span class="n">description</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">_cursor</span><span class="p">.</span><span class="n">description</span><span class="p">)}</span>
<span class="n">data</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">CursorByName</span><span class="p">(</span><span class="n">cur</span><span class="p">):</span>
<span class="n">data</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
<span class="c1">## PDF Processing
</span>
<span class="c1">## Add Header and Footer to the page
</span><span class="k">class</span> <span class="nc">PDF</span><span class="p">(</span><span class="n">FPDF</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">header</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Logo
</span> <span class="bp">self</span><span class="p">.</span><span class="n">image</span><span class="p">(</span><span class="s">'ibmi.png'</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">20</span><span class="p">)</span>
<span class="c1"># Arial bold 15
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'B'</span><span class="p">,</span> <span class="mi">15</span><span class="p">)</span>
<span class="c1"># Move to the right
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">80</span><span class="p">)</span>
<span class="c1"># Title
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">30</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="s">'Customer balance due report '</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s">'C'</span><span class="p">)</span>
<span class="c1"># Line break
</span> <span class="bp">self</span><span class="p">.</span><span class="n">ln</span><span class="p">(</span><span class="mi">40</span><span class="p">)</span>
<span class="c1"># Add dashed line
</span> <span class="bp">self</span><span class="p">.</span><span class="n">dashed_line</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">200</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="c1"># Add line break after the dashed line
</span> <span class="bp">self</span><span class="p">.</span><span class="n">ln</span><span class="p">(</span><span class="mi">20</span><span class="p">)</span>
<span class="c1"># Page footer
</span> <span class="k">def</span> <span class="nf">footer</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Position at 1.5 cm from bottom
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_y</span><span class="p">(</span><span class="o">-</span><span class="mi">15</span><span class="p">)</span>
<span class="c1"># Arial italic 8
</span> <span class="bp">self</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'I'</span><span class="p">,</span> <span class="mi">8</span><span class="p">)</span>
<span class="c1"># Page number
</span> <span class="bp">self</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="s">'Page '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">page_no</span><span class="p">()),</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s">'C'</span><span class="p">)</span>
<span class="c1"># Instatiate the PDF class and add page with attributes
</span><span class="n">pdf</span> <span class="o">=</span> <span class="n">PDF</span><span class="p">()</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">add_page</span><span class="p">()</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_left_margin</span><span class="p">(</span><span class="n">margin</span><span class="o">=</span><span class="mi">40</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">'B'</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_fill_color</span><span class="p">(</span><span class="mi">193</span><span class="p">,</span><span class="mi">229</span><span class="p">,</span><span class="mi">252</span><span class="p">)</span>
<span class="c1">#Set the table headers, with background fill color
</span><span class="n">fill_color</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">40</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Customer Number'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Last Name'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Balance Due'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="s">'Credit Limit'</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="c1"># Style set up for rows
</span><span class="n">pdf</span><span class="p">.</span><span class="n">set_font</span><span class="p">(</span><span class="s">'Arial'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">set_fill_color</span><span class="p">(</span><span class="mi">235</span><span class="p">,</span><span class="mi">236</span><span class="p">,</span><span class="mi">236</span><span class="p">)</span>
<span class="c1"># Process the database rows to print to the PDF
</span><span class="n">fill_color</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="c1"># Get data from each row of the table
</span> <span class="n">cus_num</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'CUSNUM'</span><span class="p">])</span>
<span class="n">lst_nam</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="s">'LSTNAM'</span><span class="p">]</span>
<span class="n">bal_due</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'BALDUE'</span><span class="p">])</span>
<span class="n">cdt_lmt</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="s">'CDTLMT'</span><span class="p">])</span>
<span class="c1"># Create cells with the data and print it on PDF
</span> <span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">40</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">cus_num</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">lst_nam</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'L'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">bal_due</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'R'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="n">pdf</span><span class="p">.</span><span class="n">cell</span><span class="p">(</span><span class="n">w</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">txt</span><span class="o">=</span><span class="n">cdt_lmt</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">ln</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">align</span><span class="o">=</span><span class="s">'R'</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">fill_color</span><span class="p">)</span>
<span class="c1"># Flip the fill color for alternate row.
</span> <span class="k">if</span> <span class="p">(</span><span class="n">fill_color</span><span class="o">==</span><span class="mi">0</span><span class="p">):</span>
<span class="n">fill_color</span><span class="o">=</span><span class="mi">1</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">fill_color</span><span class="o">=</span><span class="mi">0</span>
<span class="c1"># Output the PDF file
</span><span class="n">pdf</span><span class="p">.</span><span class="n">output</span><span class="p">(</span><span class="s">'customer.pdf'</span><span class="p">,</span><span class="s">'F'</span><span class="p">)</span>
</code></pre></div></div>
<h3 id="deactivate-the-virtual-environment">Deactivate the virtual environment</h3>
<p style="text-align: justify;">Once we are done running our utility, we can deactivate the virtual environment by issuing the below command from the same session. This puts us back to use the system’s default Python interpreter with all its installed libraries.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>deactivate
</code></pre></div></div>
<h3 id="how-to-install-and-run-the-utility">How to install and run the utility</h3>
<ol>
<li>Download the utility folder from <a href="https://github.com/anand-khekale/ibmi-pdf/archive/main.zip">here</a>.</li>
<li>In IFS create directory <code class="language-plaintext highlighter-rouge">ibmi-pdf</code>.</li>
<li>Upload the content of the folder in IFS in directory <code class="language-plaintext highlighter-rouge">ibmi-pdf</code></li>
<li>Login to SSH or CALL QP2TERM session.</li>
<li>Change the current directory to <code class="language-plaintext highlighter-rouge">ibmi-pdf</code> using <code class="language-plaintext highlighter-rouge">cd ibmi-pdf</code>.</li>
<li>Create virtual environment using <code class="language-plaintext highlighter-rouge">python3 -m venv --system-site-packages venv1</code>. (This will take some time, and is a one-time activity.)</li>
<li>Activate the virtual environment by using <code class="language-plaintext highlighter-rouge">. venv1/bin/activate</code>.</li>
<li>Install the FPDF package using <code class="language-plaintext highlighter-rouge">pip3 install fpdf</code>.</li>
<li>Run the utility by using <code class="language-plaintext highlighter-rouge">python3 genpdf.py</code>.</li>
<li>You should have <code class="language-plaintext highlighter-rouge">customer.pdf</code> file ready upon successful run of the utility.</li>
<li>Deactivate the virtual environment using <code class="language-plaintext highlighter-rouge">deactivate</code>.</li>
</ol>
<h3 id="special-thanks">Special Thanks</h3>
<ol>
<li><a href="http://pub400.COM">PUB400.COM</a> - Such a fantastic server to try out new things on the IBM i.</li>
<li><a href="http://papaya.io">Papaya.io</a> - Web-based image editing tool, very easy and fast to edit images.</li>
</ol>
<h3 id="conclusion">Conclusion</h3>
<p style="text-align: justify;">We have seen the power of Node.js in my <a href="https://www.anandk.dev/2020/10/NodeJS-Webservice-IBMi-AS400.html">earlier articles</a>. Similarly, Python can be used to our advantage to produce PDFs, generating charts, and doing data analytics on the IBM i. I hope this article provided an introduction to how to generate PDFs using Python on the IBM i.</p>
<p style="text-align: justify;">As always, request you to get in touch with me on <a href="https://www.linkedin.com/in/anandkdev">LinkedIn</a>/<a href="https://twitter.com/anandkdev">Twitter</a> or <a href="mailto:anand.khekale@gmail.com">email</a> with your valuable feedback. Thanks for reading!</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn this article, let’s see how we can use Python on the IBM i to generate well-formatted PDFs. The advantage of using Python for this is, it provides various packages so that the formatting, placement of the tables, etc. are handled easily; we will be using one of such packages. I have assumed a basic knowledge of Python, but have given enough details for the beginners. Let’s see how we can start using Python on the IBM i.Create beautiful Excel using exceljs (Node.js) on IBM i2020-11-29T00:00:00-06:002020-11-29T00:00:00-06:00https://www.anandk.dev/2020/11/Create-excel-on-ibmi<p style="text-align: justify;"><img src="https://www.anandk.dev/assets/images/forposts/ibmi-exceljs/ibmi-exceljs-header.png" alt="ibmi-excel-header" class="align-center" />
In this article, we will see how to create advanced Excel sheets on the IBM i using exceljs/Node.js.
We know how to create a simple extract of database files, we know how we can download a physical file using the ACS (Client Access) tool. But how about automating the extract of information needed by the user and provide a beautiful ready to use Excel extract?</p>
<p style="text-align: justify;">Many users request raw data in Excel or CSV format and then slice and dice it for higher management consumption; they spend a lot of time doing that. If we can provide them with the final version of their desired extract, I am sure the users will be happy with us in the IT department. I like to see a satisfied end-user.</p>
<p style="text-align: justify;">Let’s see how to extract data in Excel, and add some simple formatting and some conditional formatting?</p>
<blockquote>
<p>Note: The code for this utility can be downloaded from <a href="https://github.com/anand-khekale/ibmi-excel/archive/main.zip">here</a>.</p>
</blockquote>
<h3 id="how-to-generate-a-simple-excel-sheet">How to generate a simple Excel sheet</h3>
<p style="text-align: justify;">Let’s begin with creating a simple Excel worksheet using a Node.js application. We will use the default table available on the IBM i - <code class="language-plaintext highlighter-rouge">QIWS/QCUSTCDT</code>.</p>
<p>We are using two Node.js libraries.</p>
<ul>
<li>idb-pconnector - For connecting to DB2 and extracting data</li>
<li>exceljs - For creating Excel files</li>
</ul>
<p>Kindly refer to my earlier <a href="https://www.anandk.dev/2020/10/NodeJS-Webservice-IBMi-AS400.html#how-to-start-writing-nodejs-application">article</a> on the initial steps needed to initialize a Node.js application. Here I am giving high-level steps. These commands need to be invoked using an SSH session or <code class="language-plaintext highlighter-rouge">CALL QP2TERM</code> session on the IBM i.</p>
<ol>
<li>Create a directory for the application in IFS, under your home/UserID directory <code class="language-plaintext highlighter-rouge">mkdir ibmi-excel</code> .</li>
<li>Change the current directory to the above directory <code class="language-plaintext highlighter-rouge">cd ibmi-excel</code>.</li>
<li>Initialize the Node.js application using the <code class="language-plaintext highlighter-rouge">npm init</code> command, which will generate a <code class="language-plaintext highlighter-rouge">package.json</code> file.</li>
<li>Install both the libraries using <code class="language-plaintext highlighter-rouge">npm install idb-pconnector exceljs</code>, this will generate <code class="language-plaintext highlighter-rouge">package-lock.json</code> file.</li>
</ol>
<p>After the libraries are installed, let’s start writing our code to generate a simple Excel sheet.</p>
<p>Here is the code that will do the job for us, comments are included in it.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="p">{</span> <span class="nx">Connection</span><span class="p">,</span> <span class="nx">Statement</span><span class="p">,</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">idb-pconnector</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">Excel</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">exceljs</span><span class="dl">'</span><span class="p">);</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">generateExcel</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Create connection with DB2</span>
<span class="kd">const</span> <span class="nx">connection</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Connection</span><span class="p">({</span> <span class="na">url</span><span class="p">:</span> <span class="dl">'</span><span class="s1">*LOCAL</span><span class="dl">'</span> <span class="p">});</span>
<span class="kd">const</span> <span class="nx">statement</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Statement</span><span class="p">(</span><span class="nx">connection</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">sql</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">SELECT CUSNUM, LSTNAM, BALDUE, CDTLMT FROM QIWS.QCUSTCDT</span><span class="dl">'</span>
<span class="c1">// Execute the statement to fetch data in results</span>
<span class="kd">const</span> <span class="nx">results</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">exec</span><span class="p">(</span><span class="nx">sql</span><span class="p">);</span>
<span class="c1">// Create Excel workbook and worksheet</span>
<span class="kd">const</span> <span class="nx">workbook</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Excel</span><span class="p">.</span><span class="nx">Workbook</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">worksheet</span> <span class="o">=</span> <span class="nx">workbook</span><span class="p">.</span><span class="nx">addWorksheet</span><span class="p">(</span><span class="dl">'</span><span class="s1">Customers</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// Define columns in the worksheet, these columns are identified using a key.</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">columns</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Id</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">CUSNUM</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Last Name</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">LSTNAM</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Balance Due</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">BALDUE</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">11</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Credit Limit</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">CDTLMT</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">}</span>
<span class="p">]</span>
<span class="c1">// Add rows from database to worksheet </span>
<span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">row</span> <span class="k">of</span> <span class="nx">results</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">addRow</span><span class="p">(</span><span class="nx">row</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Finally save the worksheet into the folder from where we are running the code. </span>
<span class="k">await</span> <span class="nx">workbook</span><span class="p">.</span><span class="nx">xlsx</span><span class="p">.</span><span class="nx">writeFile</span><span class="p">(</span><span class="dl">'</span><span class="s1">SimpleCust.xlsx</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">generateExcel</span><span class="p">().</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>This program will generate a simple Excel like the below.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-exceljs/simple-cust.png" alt="simple-excel-extract" class="align-center" /></p>
<p style="text-align: justify;">As you can see, this is a very raw format; let us start enhancing this Excel. The code to connect to the database and extracting the Results array remains the same. We will see Excel specific code below.</p>
<h3 id="how-to-add-some-level-of-formatting-to-the-excel">How to add some level of formatting to the Excel</h3>
<p>How about adding a filter to the header row? A one-line code as given below does that.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Add autofilter on each column</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">autoFilter</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">A1:D1</span><span class="dl">'</span><span class="p">;</span>
</code></pre></div></div>
<p>What if I want to have a background color for the header row and borders for all the cells? Sure, see below.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">.......</span>
<span class="c1">// Add rows from database to worksheet </span>
<span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">row</span> <span class="k">of</span> <span class="nx">results</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">addRow</span><span class="p">(</span><span class="nx">row</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Add autofilter on each column</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">autoFilter</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">A1:D1</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// Process each row for beautification </span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">eachRow</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">row</span><span class="p">,</span> <span class="nx">rowNumber</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">row</span><span class="p">.</span><span class="nx">eachCell</span><span class="p">((</span><span class="nx">cell</span><span class="p">,</span> <span class="nx">colNumber</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">rowNumber</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// First set the background of header row</span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">fill</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">type</span><span class="p">:</span> <span class="dl">'</span><span class="s1">pattern</span><span class="dl">'</span><span class="p">,</span>
<span class="na">pattern</span><span class="p">:</span> <span class="dl">'</span><span class="s1">solid</span><span class="dl">'</span><span class="p">,</span>
<span class="na">fgColor</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">f5b914</span><span class="dl">'</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">// Set border of each cell </span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">border</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">top</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">left</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">bottom</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">right</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">}</span>
<span class="p">};</span>
<span class="p">})</span>
<span class="c1">//Commit the changed row to the stream</span>
<span class="nx">row</span><span class="p">.</span><span class="nx">commit</span><span class="p">();</span>
<span class="p">});</span>
<span class="p">...........</span>
</code></pre></div></div>
<p>We get the below spreadsheet once we run the above code. Much better. Isn’t it?</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-exceljs/formatted-cust.png" alt="formatted-excel-extract" class="align-center" /></p>
<h3 id="how-to-add-conditional-formatting">How to add conditional formatting</h3>
<p style="text-align: justify;">How about adding some conditional formatting? Let’s say the user wants Balance Due highlighted if it is 400 or more. Let us see how we can accommodate this request.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">.........</span>
<span class="nx">row</span><span class="p">.</span><span class="nx">commit</span><span class="p">();</span>
<span class="p">});</span>
<span class="c1">//Process each column for conditioning </span>
<span class="kd">const</span> <span class="nx">balDue</span> <span class="o">=</span> <span class="nx">worksheet</span><span class="p">.</span><span class="nx">getColumn</span><span class="p">(</span><span class="dl">'</span><span class="s1">BALDUE</span><span class="dl">'</span><span class="p">)</span>
<span class="c1">// iterate over all current cells in this column</span>
<span class="nx">balDue</span><span class="p">.</span><span class="nx">eachCell</span><span class="p">((</span><span class="nx">cell</span><span class="p">,</span> <span class="nx">rowNumber</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// If the balance due is 400 or more, highlight it with gradient color</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">cell</span><span class="p">.</span><span class="nx">value</span> <span class="o">>=</span> <span class="mi">400</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">fill</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">type</span><span class="p">:</span> <span class="dl">'</span><span class="s1">gradient</span><span class="dl">'</span><span class="p">,</span>
<span class="na">gradient</span><span class="p">:</span> <span class="dl">'</span><span class="s1">angle</span><span class="dl">'</span><span class="p">,</span>
<span class="na">degree</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="na">stops</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ffffff</span><span class="dl">'</span> <span class="p">}</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mf">0.5</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">cc8188</span><span class="dl">'</span> <span class="p">}</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">fa071e</span><span class="dl">'</span> <span class="p">}</span> <span class="p">}</span>
<span class="p">]</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="p">......</span>
</code></pre></div></div>
<p style="text-align: justify;">With the above code, we get our final beautiful Excel. We can further enhance it by adding calculations, tables, etc.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/ibmi-exceljs/final-cust.png" alt="final-beautiful-excel" class="align-center" /></p>
<p>The complete code is as below.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="p">{</span> <span class="nx">Connection</span><span class="p">,</span> <span class="nx">Statement</span><span class="p">,</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">idb-pconnector</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">Excel</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">exceljs</span><span class="dl">'</span><span class="p">);</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">generateExcel</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Create connection with DB2</span>
<span class="kd">const</span> <span class="nx">connection</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Connection</span><span class="p">({</span> <span class="na">url</span><span class="p">:</span> <span class="dl">'</span><span class="s1">*LOCAL</span><span class="dl">'</span> <span class="p">});</span>
<span class="kd">const</span> <span class="nx">statement</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Statement</span><span class="p">(</span><span class="nx">connection</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">sql</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">SELECT CUSNUM, LSTNAM, BALDUE, CDTLMT FROM QIWS.QCUSTCDT</span><span class="dl">'</span>
<span class="c1">// Execute the statement to fetch data in results</span>
<span class="kd">const</span> <span class="nx">results</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">exec</span><span class="p">(</span><span class="nx">sql</span><span class="p">);</span>
<span class="c1">// Create Excel workbook and worksheet</span>
<span class="kd">const</span> <span class="nx">workbook</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Excel</span><span class="p">.</span><span class="nx">Workbook</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">worksheet</span> <span class="o">=</span> <span class="nx">workbook</span><span class="p">.</span><span class="nx">addWorksheet</span><span class="p">(</span><span class="dl">'</span><span class="s1">Customers</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// Define columns in the worksheet, these columns are identified using a key.</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">columns</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">id</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">CUSNUM</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Last Name</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">LSTNAM</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Balance Due</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">BALDUE</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">11</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">header</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Credit Limit</span><span class="dl">'</span><span class="p">,</span> <span class="na">key</span><span class="p">:</span> <span class="dl">'</span><span class="s1">CDTLMT</span><span class="dl">'</span><span class="p">,</span> <span class="na">width</span><span class="p">:</span> <span class="mi">10</span> <span class="p">}</span>
<span class="p">];</span>
<span class="c1">// Add rows from database to worksheet </span>
<span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">row</span> <span class="k">of</span> <span class="nx">results</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">addRow</span><span class="p">(</span><span class="nx">row</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Add auto-filter on each column</span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">autoFilter</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">A1:D1</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// Process each row for calculations and beautification </span>
<span class="nx">worksheet</span><span class="p">.</span><span class="nx">eachRow</span><span class="p">((</span><span class="nx">row</span><span class="p">,</span> <span class="nx">rowNumber</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">row</span><span class="p">.</span><span class="nx">eachCell</span><span class="p">((</span><span class="nx">cell</span><span class="p">,</span> <span class="nx">colNumber</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">rowNumber</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// First set the background of header row</span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">fill</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">type</span><span class="p">:</span> <span class="dl">'</span><span class="s1">pattern</span><span class="dl">'</span><span class="p">,</span>
<span class="na">pattern</span><span class="p">:</span> <span class="dl">'</span><span class="s1">solid</span><span class="dl">'</span><span class="p">,</span>
<span class="na">fgColor</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">f5b914</span><span class="dl">'</span> <span class="p">}</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="c1">// Set border of each cell </span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">border</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">top</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">left</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">bottom</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">},</span>
<span class="na">right</span><span class="p">:</span> <span class="p">{</span> <span class="na">style</span><span class="p">:</span> <span class="dl">'</span><span class="s1">thin</span><span class="dl">'</span> <span class="p">}</span>
<span class="p">};</span>
<span class="p">})</span>
<span class="c1">//Commit the changed row to the stream</span>
<span class="nx">row</span><span class="p">.</span><span class="nx">commit</span><span class="p">();</span>
<span class="p">});</span>
<span class="c1">//Process 'Balance Due' column for conditioning </span>
<span class="kd">const</span> <span class="nx">balDue</span> <span class="o">=</span> <span class="nx">worksheet</span><span class="p">.</span><span class="nx">getColumn</span><span class="p">(</span><span class="dl">'</span><span class="s1">BALDUE</span><span class="dl">'</span><span class="p">)</span>
<span class="c1">// Iterate over all current cells in this column</span>
<span class="nx">balDue</span><span class="p">.</span><span class="nx">eachCell</span><span class="p">((</span><span class="nx">cell</span><span class="p">,</span> <span class="nx">rowNumber</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// If the balance due is 400 or more, highlight it with gradient color </span>
<span class="k">if</span> <span class="p">(</span><span class="nx">cell</span><span class="p">.</span><span class="nx">value</span> <span class="o">>=</span> <span class="mi">400</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">cell</span><span class="p">.</span><span class="nx">fill</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">type</span><span class="p">:</span> <span class="dl">'</span><span class="s1">gradient</span><span class="dl">'</span><span class="p">,</span>
<span class="na">gradient</span><span class="p">:</span> <span class="dl">'</span><span class="s1">angle</span><span class="dl">'</span><span class="p">,</span>
<span class="na">degree</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="na">stops</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ffffff</span><span class="dl">'</span> <span class="p">}</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mf">0.5</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">cc8188</span><span class="dl">'</span> <span class="p">}</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">position</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="na">color</span><span class="p">:</span> <span class="p">{</span> <span class="na">argb</span><span class="p">:</span> <span class="dl">'</span><span class="s1">fa071e</span><span class="dl">'</span> <span class="p">}</span> <span class="p">}</span>
<span class="p">]</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="p">});</span>
<span class="c1">// Write the final Excel file in the folder from where we are running the code. </span>
<span class="k">await</span> <span class="nx">workbook</span><span class="p">.</span><span class="nx">xlsx</span><span class="p">.</span><span class="nx">writeFile</span><span class="p">(</span><span class="dl">'</span><span class="s1">Customers.xlsx</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Call the generateExcel function</span>
<span class="nx">generateExcel</span><span class="p">().</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<h3 id="how-to-install-and-run-this-utility">How to install and run this utility?</h3>
<p>Here are the steps on how you can install this utility and test it.</p>
<ol>
<li>Download the application <a href="https://github.com/anand-khekale/ibmi-excel/archive/main.zip">folder</a> on your Laptop/PC.</li>
<li>Create a folder in IFS <code class="language-plaintext highlighter-rouge">mkdir ibmi-excel</code></li>
<li>Upload all the files using the ACS IFS upload option. (Or directly clone it if you have git on IBM i)</li>
<li>Login to SSH session or from the green screen command line <code class="language-plaintext highlighter-rouge">CALL QP2TERM</code></li>
<li>Change your current directory to the above folder <code class="language-plaintext highlighter-rouge">cd ibmi-excel</code></li>
<li>Install the dependencies by running the command <code class="language-plaintext highlighter-rouge">npm install</code></li>
<li>Once the libraries are installed, call the utility with command <code class="language-plaintext highlighter-rouge">node final.js</code></li>
<li>You should have the<code class="language-plaintext highlighter-rouge">Customers.xlsx</code> file ready upon a successful run of the program.</li>
</ol>
<h3 id="further-reading">Further Reading</h3>
<ol>
<li>Exceljs Documentation - <a href="https://www.npmjs.com/package/exceljs">Link</a></li>
<li>idb-pconnector documentation - <a href="https://www.npmjs.com/package/idb-pconnector">Link</a></li>
</ol>
<h3 id="special-thanks">Special Thanks</h3>
<ol>
<li><a href="http://pub400.COM">PUB400.COM</a> - Such a fantastic server to try out new things on the IBM i.</li>
<li><a href="http://papaya.io">Papaya.io</a> - Web-based image editing tool, very easy and fast to edit images.</li>
</ol>
<h3 id="conclusion">Conclusion:</h3>
<p style="text-align: justify;">With the open-source languages available on the IBM i, the possibilities of creating beautiful content for the end-users are endless. With little effort, we can deliver the latest output whether it be a web screen, reports formatted in PDF or simple data extracts like the above.</p>
<p>Hope this showcased how easy it is to build an extract with Node.js.</p>
<p style="text-align: justify;">As always, request you to get in touch with me on <a href="https://www.linkedin.com/in/anandkdev">LinkedIn</a>/<a href="https://twitter.com/anandkdev">Twitter</a> or <a href="mailto:anand.khekale@gmail.com">email</a> with your valuable feedback. Thanks for reading!</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn this article, we will see how to create advanced Excel sheets on the IBM i using exceljs/Node.js. We know how to create a simple extract of database files, we know how we can download a physical file using the ACS (Client Access) tool. But how about automating the extract of information needed by the user and provide a beautiful ready to use Excel extract? Many users request raw data in Excel or CSV format and then slice and dice it for higher management consumption; they spend a lot of time doing that. If we can provide them with the final version of their desired extract, I am sure the users will be happy with us in the IT department. I like to see a satisfied end-user. Let’s see how to extract data in Excel, and add some simple formatting and some conditional formatting?How to use Visual Studio Code while working on IBM i2020-11-02T00:00:00-06:002020-11-02T00:00:00-06:00https://www.anandk.dev/2020/11/VSCode-IBMi-AS400<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_header.png" alt="vscode-ibmi" class="align-center" /></p>
<p style="text-align: justify;">In this article, let’s see how we can use the Visual Studio Code (VS Code) while working on IBM i. This versatile editor is used for writing programs with languages like Java, JavaScript, Python, and many more, but how we can leverage it for writing RPGLE, SQLRPGLE, CLLE code is also important.</p>
<p style="text-align: justify;">In recent months VS Code has emerged as the most preferred code editor in the programming fraternity. It is an open-source, free, lightweight code editor from Microsoft. This editor has tons of extensions, themes, customization, and powerful language editing features.</p>
<p style="text-align: justify;">This article also covers the extensions, themes, and settings needed for the program to be written directly on the IBM i system, for its final compilation/deployment.</p>
<blockquote>
<p>Note: The code for this article can be downloaded from <a href="https://github.com/anand-khekale/vscode-ibmi/archive/master.zip">here</a>.</p>
</blockquote>
<h2 id="how-to-install-vs-code">How to install VS Code</h2>
<p style="text-align: justify;">Visit the <a href="https://code.visualstudio.com/">Visual Studio Code</a> site, click on the download button for your operating system, once downloaded, install it like any other software program; it is easy and straight forward. I use their stable version.</p>
<p>There is detailed documentation on how to <a href="https://code.visualstudio.com/docs">get started</a> on it.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_download.png" alt="vscode-ibmi" class="align-center" /></p>
<h2 id="extensions-in-vs-code">Extensions in VS Code</h2>
<p style="text-align: justify;">VS Code provides many extensions that help improve code writing efficiency. I have given some of the extensions below to showcase its usefulness.</p>
<h3 id="1-ibm-i-languages-by-barrettotte">1. IBM i Languages by barrettotte</h3>
<p style="text-align: justify;"><a href="https://marketplace.visualstudio.com/items?itemName=barrettotte.ibmi-languages">IBM i Languages</a> is an extension for the majority of native languages like RPG, CL, DDS, and RPGLE fixed/free. This extension provides syntax highlighting, keyword prompting, etc. See the snapshots below, where all the opcodes are highlighted.</p>
<p>This is for RPGLE Free.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ibmi_rpgle.png" alt="vscode-ibmi" class="align-center" /></p>
<p>This is for CLLE.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ibmi_clle.png" alt="vscode-ibmi" class="align-center" /></p>
<p>And one for DDS.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ibmi_dds.png" alt="vscode-ibmi" class="align-center" /></p>
<h3 id="2-ibm-z-open-editor">2. IBM Z Open Editor</h3>
<p style="text-align: justify;">This <a href="https://marketplace.visualstudio.com/items?itemName=IBM.zopeneditor">COBOL</a> extension is so powerful, with few keystrokes, most of your program gets generated. Let’s see, when I start typing IDENT and hit Enter, the entire IDENTIFICATION DIVISION gets generated using the snippets provided by the extension.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_cobol.gif" alt="vscode-ibmi" class="align-center" /></p>
<h3 id="3-extensionsthemes-for-open-source-languages">3. Extensions/Themes for Open-source Languages</h3>
<p>The below table summarizes some of the extensions/themes I use, which help me a lot when writing a program.</p>
<table>
<thead>
<tr>
<th>Extension</th>
<th style="text-align: left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>Bracket Pair Colorizer</td>
<td style="text-align: left">A customizable extension for colorizing matching brackets - <a href="https://marketplace.visualstudio.com/items?itemName=CoenraadS.bracket-pair-colorizer">Link</a>.</td>
</tr>
<tr>
<td>Code Spell Checker</td>
<td style="text-align: left">Spelling checker for source code - <a href="https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker">Link</a>.</td>
</tr>
<tr>
<td>Indenticator</td>
<td style="text-align: left">Highlights your current indent depth - <a href="https://marketplace.visualstudio.com/items?itemName=SirTori.indenticator">Link</a>.</td>
</tr>
<tr>
<td>One Dark Pro Theme</td>
<td style="text-align: left">Atom’s iconic One Dark theme for Visual Studio Code - <a href="https://marketplace.visualstudio.com/items?itemName=zhuangtongfa.Material-theme">Link</a>.</td>
</tr>
<tr>
<td>Python</td>
<td style="text-align: left">Linting, Debugging (multi-threaded, remote), Intellisense, Jupyter Notebooks, code formatting - <a href="https://marketplace.visualstudio.com/items?itemName=ms-python.python">Link</a>.</td>
</tr>
<tr>
<td>SSH FS</td>
<td style="text-align: left">File system provider using SSH. - <a href="https://marketplace.visualstudio.com/items?itemName=Kelvin.vscode-sshfs">Link</a>.</td>
</tr>
<tr>
<td>Visual Studio IntelliCode</td>
<td style="text-align: left">AI-assisted development. - <a href="https://marketplace.visualstudio.com/items?itemName=VisualStudioExptTeam.vscodeintellicode">Link</a>.</td>
</tr>
<tr>
<td>vscode-icons</td>
<td style="text-align: left">Icons for Visual Studio Code. - <a href="https://marketplace.visualstudio.com/items?itemName=vscode-icons-team.vscode-icons">Link</a>.</td>
</tr>
</tbody>
</table>
<p style="text-align: justify;">These extensions help syntax highlighting and aid in errorless code.
Below is an example of how JavaScript code looks in the VS Code. Beautiful, isn’t it?</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_javascript.png" alt="vscode-ibmi" class="align-center" /></p>
<h2 id="how-to-install-extensions">How to install Extensions</h2>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_extension.png" alt="vscode-ibmi" class="align-right" />Click on the extension symbol as displayed by the image, type the desired extension in the search bar, when the extension appears, click on the install button, and you are done. More information <a href="https://code.visualstudio.com/docs/editor/extension-gallery">here</a>.</p>
<h2 id="how-to-write-code-directly-in-ifs-on-ibm-i">How to write code directly in IFS on IBM i?</h2>
<p style="text-align: justify;">It becomes convenient if we don’t have to upload the code often to the IFS directory on IBM i manually. One extension always comes in handy that is SSH-FS, which makes the IFS folder available to us in the VS Code.</p>
<h3 id="how-to-use-ssh-fs">How to use SSH-FS</h3>
<p style="text-align: justify;">Install the extension, as explained earlier. You should have a basic knowledge of how SSH works. Aaron Bartell has explained it here on <a href="https://www.itjungle.com/2014/09/17/fhg091714-story01/">IT Jungle</a>, below is a recap.</p>
<p style="text-align: justify;">Make sure that the SSH server is running on IBM i, otherwise, issue the below command from the command line on IBM i.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>STRTCPSVR SERVER<span class="o">(</span><span class="k">*</span>SSHD<span class="o">)</span>
</code></pre></div></div>
<p>Below are the steps to connect to the IBM i system from Windows Command Prompt. (I am connecting to <a href="https://pub400.com/">PUB400.com</a> server with my user-id.)</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ssh1.png" alt="vscode-ibmi" class="align-center" /></p>
<p>When prompted, enter your IBM i password.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ssh2.png" alt="vscode-ibmi" class="align-center" />
You will sign on to IBM i with a shell $ prompt.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_ssh3.png" alt="vscode-ibmi" class="align-center" /></p>
<p>After you have tested your SSH connection to the IBM i, configure the SSH-FS in VS Code. Use <code class="language-plaintext highlighter-rouge">Ctrl + Shift + P</code> on Windows or <code class="language-plaintext highlighter-rouge">Cmd + Shift + P</code> on Mac and type SSHFS in the search bar.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_sshfs_setting.png" alt="vscode-ibmi" class="align-center" /></p>
<p>SSH-FS needs the below details, along with the name of the connection.</p>
<ol>
<li>The IP/Domain name of your IBM i system</li>
<li>SSH Port (generally port 22, PUB400 has 2222 port configured for SSH)</li>
<li>User Id</li>
<li>Password</li>
<li>Root folder (/home/USERID)</li>
</ol>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_sshfs_connect.png" alt="vscode-ibmi" class="align-right" /> After the configuration is done, under SSH FILE SYSTEMS Section of VS Code sidebar, when we right-click on Connect as Workplace folder, the IFS root folder will be accessible.<br />
We are ready to write our code.</p>
<p>Below is an example of the code written using SSH FS extension, in the IFS.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_sshfs_demo.png" alt="vscode-ibmi" class="align-center" /></p>
<h3 id="write-free-rpgle-code-using-ssh-fs">Write Free RPGLE code using SSH-FS</h3>
<p style="text-align: justify;">It becomes easy to write free format RPGLE using VS Code due to the extensions discussed earlier. To expedite the writing of code, I have created a snippets file, which is another feature of VS Code editor.</p>
<h3 id="what-are-snippets">What are Snippets?</h3>
<p style="text-align: justify;">VS Code has this feature called Snippets, where we can assign a shortcut to a block of code that we write quite often. We can create our snippet for the language we want to use. We have to create a file in JSON format in <code class="language-plaintext highlighter-rouge">File → Preferences →User Snippets</code>.</p>
<p style="text-align: justify;">I have created rpgle.json so that these snippets will be available for a program file with extension .rpgle. You can download and paste the content of the file from <a href="https://github.com/anand-khekale/vscode-ibmi/archive/master.zip">here</a>.</p>
<p>Go to <code class="language-plaintext highlighter-rouge">File→Preferences→User Snippets</code></p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_snippet_path.png" alt="vscode-ibmi" class="align-center" /></p>
<p>You will be prompted with a search bar, enter <strong>rpgle.json.</strong></p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_snippet_search.png" alt="vscode-ibmi" class="align-center" /></p>
<p>Paste the content of rpgle.json file as below.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_snippet_json.png" alt="vscode-ibmi" class="align-center" /></p>
<p style="text-align: justify;">Below is the list of shortcuts to start with and these can be expanded as per our needs. → is used to denote the Tab key.</p>
<table>
<thead>
<tr>
<th>Shortcut</th>
<th style="text-align: left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>CTLOPT →</td>
<td style="text-align: left">Add Ctl-Opt at the beginning of the code</td>
</tr>
<tr>
<td>DF →</td>
<td style="text-align: left">Define File with DCL-F</td>
</tr>
<tr>
<td>DW →</td>
<td style="text-align: left">Define display file with DCL-F</td>
</tr>
<tr>
<td>DC →</td>
<td style="text-align: left">Define Character Field with DCL-S</td>
</tr>
<tr>
<td>DV →</td>
<td style="text-align: left">Define VARCHAR field</td>
</tr>
<tr>
<td>DI →</td>
<td style="text-align: left">Define Integer field</td>
</tr>
<tr>
<td>DZ →</td>
<td style="text-align: left">Define Zoned field</td>
</tr>
<tr>
<td>PSDS →</td>
<td style="text-align: left">Defind program information datastructure</td>
</tr>
<tr>
<td>BEGSR →</td>
<td style="text-align: left">Create a subroutine</td>
</tr>
<tr>
<td>CMT= →</td>
<td style="text-align: left">Add a comment with //=====//</td>
</tr>
<tr>
<td>CMT- →</td>
<td style="text-align: left">Add a comment with //——-//</td>
</tr>
<tr>
<td>DOW →</td>
<td style="text-align: left">Write a DoW loop</td>
</tr>
<tr>
<td>DOU →</td>
<td style="text-align: left">Write a DoU loop</td>
</tr>
<tr>
<td>IFENDIF →</td>
<td style="text-align: left">Write IF - Endif block</td>
</tr>
<tr>
<td>IFELSE →</td>
<td style="text-align: left">Write IF - ELSE - ENDIF block</td>
</tr>
</tbody>
</table>
<h3 id="writing-code-using-rpgle-snippets">Writing code using RPGLE Snippets</h3>
<p style="text-align: justify;">With snippets, it becomes fast and joyful to write code. You just have to use the shortcut and press the Tab key or the Enter key, and the code gets generated for you.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_snippet_demo.gif" alt="vscode-ibmi" class="align-center" /></p>
<h3 id="copying-and-compiling-the-ibm-i-code">Copying and compiling the IBM i code</h3>
<p style="text-align: justify;">When we have completed coding in IFS, it will be better if we automate copying these files to respective source physical files and compile those. It will be a burden if you have to do it manually for each program. Hence, a need for shell script comes in, which I have pasted below.</p>
<p style="text-align: justify;">This shell script will take any file from its current directory with the appropriate extension, let’s say .rpgle, and copy and compile it. For this to run, we have to use our SSH session or QP2TERM (CALL QP2TERM) session on the IBM i.</p>
<p>From the $ prompt, make the file executable for the first time as below.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">chmod</span> +x buildpgm.sh
</code></pre></div></div>
<p>And run it with the below command.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>./buildpgm.sh
</code></pre></div></div>
<p style="text-align: justify;">Well, the native IBM i programs can be compiled directly from IFS, but if you have DSPFs, those need source physical file. So for now, I keep everything in respective source physical files.</p>
<p>Below is such a script I have created, which does the job as explained.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/QOpenSys/usr/bin/sh</span>
<span class="c"># ------------------------------------------------------------------------- #</span>
<span class="c"># Program : buildpgm.sh</span>
<span class="c"># Author : www.anandk.dev</span>
<span class="c"># Date Written : 27th October 2020</span>
<span class="c"># Inspired from : https://github.com/barrettotte/RPGLE-Twilio/blob/master/build.sh</span>
<span class="c"># ------------------------------------------------------------------------- #</span>
<span class="c"># This program reads current folder from IFS and takes below steps.</span>
<span class="c"># 1. It assumes these sources i.e. dspf, rpgle, sqlrpgle, clle. </span>
<span class="c"># 2. It copies .rpgle, .sqlrpgle, .clle and .dspf to respective _SRC. </span>
<span class="c"># 3. It compiles the DSPF first, then RPGLE, SQLRPGLE and finally CLLE. </span>
<span class="c"># After coping this program to IFS folder where your sources are, use below command</span>
<span class="c"># to make this file executable. </span>
<span class="c"># chmod +x buildpgm.sh</span>
<span class="c"># Set the CUR_LIB to your library.</span>
<span class="c"># Check if the SRCPF names are correct for you. </span>
<span class="c"># Set the application name which will be applied to all the memebers as member text.</span>
<span class="c"># ------------------------------------------------------------------------- #</span>
<span class="c"># Set the IFS directory, if not given, set current</span>
<span class="nv">IFS_DIR</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">1</span><span class="k">:-</span><span class="si">$(</span><span class="nb">pwd</span><span class="si">)</span><span class="k">}</span><span class="s2">"</span>
<span class="c"># Set the current library where the programs will be compiled. </span>
<span class="nv">CUR_LIB</span><span class="o">=</span><span class="s2">"ANANDK1"</span>
<span class="c"># Set source physical files </span>
<span class="nv">DDS_SRC</span><span class="o">=</span><span class="s2">"QDDSSRC"</span>
<span class="nv">RPGLE_SRC</span><span class="o">=</span><span class="s2">"QRPGLESRC"</span>
<span class="nv">CL_SRC</span><span class="o">=</span><span class="s2">"QCLSRC"</span>
<span class="c"># Set application name</span>
<span class="nv">APPLICATION</span><span class="o">=</span><span class="s2">"Weather Application"</span>
<span class="c"># Execute the command by setting the library first</span>
exec_cmd<span class="o">(){</span>
<span class="nb">echo</span> <span class="nv">$1</span>
<span class="nv">output</span><span class="o">=</span><span class="si">$(</span>qsh <span class="nt">-c</span> <span class="s2">"liblist -a </span><span class="nv">$CUR_LIB</span><span class="s2"> ; system </span><span class="se">\"</span><span class="nv">$1</span><span class="se">\"</span><span class="s2">"</span><span class="si">)</span>
<span class="k">if</span> <span class="o">[</span> <span class="nt">-n</span> <span class="s2">"</span><span class="nv">$2</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="nv">$1</span><span class="se">\n\n</span><span class="nv">$output</span><span class="s2">"</span> <span class="o">></span> <span class="s2">"</span><span class="nv">$IFS_DIR</span><span class="s2">/</span><span class="nv">$2</span><span class="s2">.log"</span>
<span class="k">fi</span>
<span class="o">}</span>
<span class="c"># Copy DSPF to source PF and compile</span>
crt_dspf<span class="o">(){</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'\n... executing commands... '</span>
<span class="nv">filename</span><span class="o">=</span><span class="si">$(</span><span class="nb">basename</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span><span class="si">)</span>
<span class="nv">member</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">filename</span><span class="p">%.*</span><span class="k">}</span><span class="s2">"</span>
exec_cmd <span class="s2">"CHGATR OBJ('</span><span class="nv">$1</span><span class="s2">') ATR(*CCSID) VALUE(819)"</span>
exec_cmd <span class="s2">"CPYFRMSTMF FROMSTMF('</span><span class="nv">$1</span><span class="s2">') TOMBR('/QSYS.lib/</span><span class="nv">$CUR_LIB</span><span class="s2">.lib/</span><span class="nv">$DDS_SRC</span><span class="s2">.file/</span><span class="nv">$member</span><span class="s2">.mbr') MBROPT(*REPLACE)"</span>
exec_cmd <span class="s2">"CHGPFM FILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$DDS_SRC</span><span class="s2">) MBR(</span><span class="nv">$member</span><span class="s2">) SRCTYPE(DSPF) TEXT('</span><span class="nv">$APPLICATION</span><span class="s2">')"</span>
exec_cmd <span class="s2">"CRTDSPF FILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$member</span><span class="s2">) SRCFILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$DDS_SRC</span><span class="s2">)"</span> <span class="nv">$member</span>
<span class="o">}</span>
<span class="c"># Copy RPGLE to SRCPF and compile</span>
crt_rpgle<span class="o">(){</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'\n... executing commands... '</span>
<span class="nv">filename</span><span class="o">=</span><span class="si">$(</span><span class="nb">basename</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span><span class="si">)</span>
<span class="nv">member</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">filename</span><span class="p">%.*</span><span class="k">}</span><span class="s2">"</span>
exec_cmd <span class="s2">"CHGATR OBJ('</span><span class="nv">$1</span><span class="s2">') ATR(*CCSID) VALUE(819)"</span>
exec_cmd <span class="s2">"CPYFRMSTMF FROMSTMF('</span><span class="nv">$1</span><span class="s2">') TOMBR('/QSYS.lib/</span><span class="nv">$CUR_LIB</span><span class="s2">.lib/</span><span class="nv">$RPGLE_SRC</span><span class="s2">.file/</span><span class="nv">$member</span><span class="s2">.mbr') MBROPT(*REPLACE)"</span>
exec_cmd <span class="s2">"CHGPFM FILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$RPGLE_SRC</span><span class="s2">) MBR(</span><span class="nv">$member</span><span class="s2">) SRCTYPE(RPGLE) TEXT('</span><span class="nv">$APPLICATION</span><span class="s2">')"</span>
exec_cmd <span class="s2">"CRTBNDRPG PGM(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$member</span><span class="s2">) DFTACTGRP(*NO) DBGVIEW(*SOURCE)"</span> <span class="nv">$member</span>
<span class="o">}</span>
<span class="c"># Copy SQLRPGLE to SRCPF and compile</span>
crt_sqlrpgle<span class="o">(){</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'\n... executing commands... '</span>
<span class="nv">filename</span><span class="o">=</span><span class="si">$(</span><span class="nb">basename</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span><span class="si">)</span>
<span class="nv">member</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">filename</span><span class="p">%.*</span><span class="k">}</span><span class="s2">"</span>
exec_cmd <span class="s2">"CHGATR OBJ('</span><span class="nv">$1</span><span class="s2">') ATR(*CCSID) VALUE(819)"</span>
exec_cmd <span class="s2">"CPYFRMSTMF FROMSTMF('</span><span class="nv">$1</span><span class="s2">') TOMBR('/QSYS.lib/</span><span class="nv">$CUR_LIB</span><span class="s2">.lib/</span><span class="nv">$RPGLE_SRC</span><span class="s2">.file/</span><span class="nv">$member</span><span class="s2">.mbr') MBROPT(*REPLACE)"</span>
exec_cmd <span class="s2">"CHGPFM FILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$RPGLE_SRC</span><span class="s2">) MBR(</span><span class="nv">$member</span><span class="s2">) SRCTYPE(SQLRPGLE) TEXT('</span><span class="nv">$APPLICATION</span><span class="s2">')"</span>
exec_cmd <span class="s2">"CRTSQLRPGI OBJ(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$member</span><span class="s2">) SRCFILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$RPGLE_SRC</span><span class="s2">) COMMIT(*NONE) DBGVIEW(*SOURCE)"</span> <span class="nv">$member</span>
<span class="o">}</span>
<span class="c"># Copy CLLE to SRCPF and compile</span>
crt_clle<span class="o">(){</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'\n... executing commands... '</span>
<span class="nv">filename</span><span class="o">=</span><span class="si">$(</span><span class="nb">basename</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span><span class="si">)</span>
<span class="nv">member</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">filename</span><span class="p">%.*</span><span class="k">}</span><span class="s2">"</span>
exec_cmd <span class="s2">"CHGATR OBJ('</span><span class="nv">$1</span><span class="s2">') ATR(*CCSID) VALUE(819)"</span>
exec_cmd <span class="s2">"CPYFRMSTMF FROMSTMF('</span><span class="nv">$1</span><span class="s2">') TOMBR('/QSYS.lib/</span><span class="nv">$CUR_LIB</span><span class="s2">.lib/</span><span class="nv">$CL_SRC</span><span class="s2">.file/</span><span class="nv">$member</span><span class="s2">.mbr') MBROPT(*REPLACE)"</span>
exec_cmd <span class="s2">"CHGPFM FILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$CL_SRC</span><span class="s2">) MBR(</span><span class="nv">$member</span><span class="s2">) SRCTYPE(CLLE) TEXT('</span><span class="nv">$APPLICATION</span><span class="s2">')"</span>
exec_cmd <span class="s2">"CRTBNDCL PGM(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$member</span><span class="s2">) SRCFILE(</span><span class="nv">$CUR_LIB</span><span class="s2">/</span><span class="nv">$CL_SRC</span><span class="s2">) DFTACTGRP(*NO) ACTGRP(*CALLER) DBGVIEW(*SOURCE)"</span> <span class="nv">$member</span>
<span class="o">}</span>
search_dspf<span class="o">(){</span>
<span class="k">for </span>FILE <span class="k">in</span> <span class="s2">"</span><span class="nv">$IFS_DIR</span><span class="s2">"</span>/<span class="k">*</span>
<span class="k">do
</span><span class="nv">ext</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">FILE</span><span class="p">##*.</span><span class="k">}</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$ext</span> <span class="o">==</span> <span class="s1">'dspf'</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">=========== Building DSPF - </span><span class="nv">$FILE</span><span class="s2"> ============="</span>
crt_dspf <span class="nv">$FILE</span>
<span class="k">fi
done</span>
<span class="o">}</span>
search_rpgle<span class="o">(){</span>
<span class="k">for </span>FILE <span class="k">in</span> <span class="s2">"</span><span class="nv">$IFS_DIR</span><span class="s2">"</span>/<span class="k">*</span>
<span class="k">do
</span><span class="nv">ext</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">FILE</span><span class="p">##*.</span><span class="k">}</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$ext</span> <span class="o">==</span> <span class="s1">'rpgle'</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">=========== Building RPGLE - </span><span class="nv">$FILE</span><span class="s2"> ============="</span>
crt_rpgle <span class="nv">$FILE</span>
<span class="k">fi
done</span>
<span class="o">}</span>
search_sqlrpgle<span class="o">(){</span>
<span class="k">for </span>FILE <span class="k">in</span> <span class="s2">"</span><span class="nv">$IFS_DIR</span><span class="s2">"</span>/<span class="k">*</span>
<span class="k">do
</span><span class="nv">ext</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">FILE</span><span class="p">##*.</span><span class="k">}</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$ext</span> <span class="o">==</span> <span class="s1">'sqlrpgle'</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">=========== Building SQLRPGLE - </span><span class="nv">$FILE</span><span class="s2"> ============="</span>
crt_sqlrpgle <span class="nv">$FILE</span>
<span class="k">fi
done</span>
<span class="o">}</span>
search_clle<span class="o">(){</span>
<span class="k">for </span>FILE <span class="k">in</span> <span class="s2">"</span><span class="nv">$IFS_DIR</span><span class="s2">"</span>/<span class="k">*</span>
<span class="k">do
</span><span class="nv">ext</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">FILE</span><span class="p">##*.</span><span class="k">}</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$ext</span> <span class="o">==</span> <span class="s1">'clle'</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">=========== Building CLLE - </span><span class="nv">$FILE</span><span class="s2"> ============="</span>
crt_clle <span class="nv">$FILE</span>
<span class="k">fi
done</span>
<span class="o">}</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"|=== Starting to build the programs for </span><span class="nv">$APPLICATION</span><span class="s2"> ===|"</span>
search_dspf
search_rpgle
search_sqlrpgle
search_clle
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'|=================================================================================|'</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'| Program build completed, please check if you encountered any errors. |'</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'| Check the log files created in the below folder. |'</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"| </span><span class="nv">$IFS_DIR</span><span class="s2"> |"</span>
<span class="nb">echo</span> <span class="nt">-e</span> <span class="s1">'|=================================================================================|'</span>
</code></pre></div></div>
<h2 id="synchronizing-your-set-up">Synchronizing your set up</h2>
<p style="text-align: justify;">It is necessary that, when you have completed all your customization for VS Code, you should sync your setup so that, if you change your laptop/desktop, those are retained and you don’t have to do it all over again.</p>
<p style="text-align: justify;">When you go to <code class="language-plaintext highlighter-rouge">File → Preferences →Setting Sync</code> you can configure where you want to sync it. I have currently set it to sync with my GitHub account.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/vscode-ibmi/vscode_setting_sync.png" alt="vscode-ibmi" class="align-center" /></p>
<h2 id="wrapping-up">Wrapping up…</h2>
<p style="text-align: justify;">I find the VS Code editor highly customizable and efficient to use while I work on IBM i. It is also available on all operating systems.</p>
<p style="text-align: justify;">I hope this article has given you some ideas on how to use the editor.</p>
<p style="text-align: justify;">As always, request you to get in touch with me on <a href="https://www.linkedin.com/in/anandkdev">LinkedIn</a>/<a href="https://twitter.com/anandkdev">Twitter</a> or <a href="mailto:anand.khekale@gmail.com">email</a> with your valuable feedback. Thanks for reading!</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn this article, let’s see how we can use the Visual Studio Code (VS Code) while working on IBM i. This versatile editor is used for writing programs with languages like Java, JavaScript, Python, and many more, but how we can leverage it for writing RPGLE, SQLRPGLE, CLLE code is also important. In recent months VS Code has emerged as the most preferred code editor in the programming fraternity. It is an open-source, free, lightweight code editor from Microsoft. This editor has tons of extensions, themes, customization, and powerful language editing features. This article also covers the extensions, themes, and settings needed for the program to be written directly on the IBM i system, for its final compilation/deployment. Note: The code for this article can be downloaded from here. How to install VS Code Visit the Visual Studio Code site, click on the download button for your operating system, once downloaded, install it like any other software program; it is easy and straight forward. I use their stable version. There is detailed documentation on how to get started on it. Extensions in VS Code VS Code provides many extensions that help improve code writing efficiency. I have given some of the extensions below to showcase its usefulness. 1. IBM i Languages by barrettotte IBM i Languages is an extension for the majority of native languages like RPG, CL, DDS, and RPGLE fixed/free. This extension provides syntax highlighting, keyword prompting, etc. See the snapshots below, where all the opcodes are highlighted. This is for RPGLE Free. This is for CLLE. And one for DDS. 2. IBM Z Open Editor This COBOL extension is so powerful, with few keystrokes, most of your program gets generated. Let’s see, when I start typing IDENT and hit Enter, the entire IDENTIFICATION DIVISION gets generated using the snippets provided by the extension. 3. Extensions/Themes for Open-source Languages The below table summarizes some of the extensions/themes I use, which help me a lot when writing a program. Extension Description Bracket Pair Colorizer A customizable extension for colorizing matching brackets - Link. Code Spell Checker Spelling checker for source code - Link. Indenticator Highlights your current indent depth - Link. One Dark Pro Theme Atom’s iconic One Dark theme for Visual Studio Code - Link. Python Linting, Debugging (multi-threaded, remote), Intellisense, Jupyter Notebooks, code formatting - Link. SSH FS File system provider using SSH. - Link. Visual Studio IntelliCode AI-assisted development. - Link. vscode-icons Icons for Visual Studio Code. - Link. These extensions help syntax highlighting and aid in errorless code. Below is an example of how JavaScript code looks in the VS Code. Beautiful, isn’t it? How to install Extensions Click on the extension symbol as displayed by the image, type the desired extension in the search bar, when the extension appears, click on the install button, and you are done. More information here. How to write code directly in IFS on IBM i? It becomes convenient if we don’t have to upload the code often to the IFS directory on IBM i manually. One extension always comes in handy that is SSH-FS, which makes the IFS folder available to us in the VS Code. How to use SSH-FS Install the extension, as explained earlier. You should have a basic knowledge of how SSH works. Aaron Bartell has explained it here on IT Jungle, below is a recap. Make sure that the SSH server is running on IBM i, otherwise, issue the below command from the command line on IBM i. STRTCPSVR SERVER(*SSHD) Below are the steps to connect to the IBM i system from Windows Command Prompt. (I am connecting to PUB400.com server with my user-id.) When prompted, enter your IBM i password. You will sign on to IBM i with a shell $ prompt. After you have tested your SSH connection to the IBM i, configure the SSH-FS in VS Code. Use Ctrl + Shift + P on Windows or Cmd + Shift + P on Mac and type SSHFS in the search bar. SSH-FS needs the below details, along with the name of the connection. The IP/Domain name of your IBM i system SSH Port (generally port 22, PUB400 has 2222 port configured for SSH) User Id Password Root folder (/home/USERID) After the configuration is done, under SSH FILE SYSTEMS Section of VS Code sidebar, when we right-click on Connect as Workplace folder, the IFS root folder will be accessible. We are ready to write our code. Below is an example of the code written using SSH FS extension, in the IFS. Write Free RPGLE code using SSH-FS It becomes easy to write free format RPGLE using VS Code due to the extensions discussed earlier. To expedite the writing of code, I have created a snippets file, which is another feature of VS Code editor. What are Snippets? VS Code has this feature called Snippets, where we can assign a shortcut to a block of code that we write quite often. We can create our snippet for the language we want to use. We have to create a file in JSON format in File → Preferences →User Snippets. I have created rpgle.json so that these snippets will be available for a program file with extension .rpgle. You can download and paste the content of the file from here. Go to File→Preferences→User Snippets You will be prompted with a search bar, enter rpgle.json. Paste the content of rpgle.json file as below. Below is the list of shortcuts to start with and these can be expanded as per our needs. → is used to denote the Tab key. Shortcut Description CTLOPT → Add Ctl-Opt at the beginning of the code DF → Define File with DCL-F DW → Define display file with DCL-F DC → Define Character Field with DCL-S DV → Define VARCHAR field DI → Define Integer field DZ → Define Zoned field PSDS → Defind program information datastructure BEGSR → Create a subroutine CMT= → Add a comment with //=====// CMT- → Add a comment with //——-// DOW → Write a DoW loop DOU → Write a DoU loop IFENDIF → Write IF - Endif block IFELSE → Write IF - ELSE - ENDIF block Writing code using RPGLE Snippets With snippets, it becomes fast and joyful to write code. You just have to use the shortcut and press the Tab key or the Enter key, and the code gets generated for you. Copying and compiling the IBM i code When we have completed coding in IFS, it will be better if we automate copying these files to respective source physical files and compile those. It will be a burden if you have to do it manually for each program. Hence, a need for shell script comes in, which I have pasted below. This shell script will take any file from its current directory with the appropriate extension, let’s say .rpgle, and copy and compile it. For this to run, we have to use our SSH session or QP2TERM (CALL QP2TERM) session on the IBM i. From the $ prompt, make the file executable for the first time as below. $ chmod +x buildpgm.sh And run it with the below command. $ ./buildpgm.sh Well, the native IBM i programs can be compiled directly from IFS, but if you have DSPFs, those need source physical file. So for now, I keep everything in respective source physical files. Below is such a script I have created, which does the job as explained. #!/QOpenSys/usr/bin/sh # ------------------------------------------------------------------------- # # Program : buildpgm.sh # Author : www.anandk.dev # Date Written : 27th October 2020 # Inspired from : https://github.com/barrettotte/RPGLE-Twilio/blob/master/build.sh # ------------------------------------------------------------------------- # # This program reads current folder from IFS and takes below steps. # 1. It assumes these sources i.e. dspf, rpgle, sqlrpgle, clle. # 2. It copies .rpgle, .sqlrpgle, .clle and .dspf to respective _SRC. # 3. It compiles the DSPF first, then RPGLE, SQLRPGLE and finally CLLE. # After coping this program to IFS folder where your sources are, use below command # to make this file executable. # chmod +x buildpgm.sh # Set the CUR_LIB to your library. # Check if the SRCPF names are correct for you. # Set the application name which will be applied to all the memebers as member text. # ------------------------------------------------------------------------- # # Set the IFS directory, if not given, set current IFS_DIR="${1:-$(pwd)}" # Set the current library where the programs will be compiled. CUR_LIB="ANANDK1" # Set source physical files DDS_SRC="QDDSSRC" RPGLE_SRC="QRPGLESRC" CL_SRC="QCLSRC" # Set application name APPLICATION="Weather Application" # Execute the command by setting the library first exec_cmd(){ echo $1 output=$(qsh -c "liblist -a $CUR_LIB ; system \"$1\"") if [ -n "$2" ]; then echo -e "$1\n\n$output" > "$IFS_DIR/$2.log" fi } # Copy DSPF to source PF and compile crt_dspf(){ echo -e '\n... executing commands... ' filename=$(basename "$1") member="${filename%.*}" exec_cmd "CHGATR OBJ('$1') ATR(*CCSID) VALUE(819)" exec_cmd "CPYFRMSTMF FROMSTMF('$1') TOMBR('/QSYS.lib/$CUR_LIB.lib/$DDS_SRC.file/$member.mbr') MBROPT(*REPLACE)" exec_cmd "CHGPFM FILE($CUR_LIB/$DDS_SRC) MBR($member) SRCTYPE(DSPF) TEXT('$APPLICATION')" exec_cmd "CRTDSPF FILE($CUR_LIB/$member) SRCFILE($CUR_LIB/$DDS_SRC)" $member } # Copy RPGLE to SRCPF and compile crt_rpgle(){ echo -e '\n... executing commands... ' filename=$(basename "$1") member="${filename%.*}" exec_cmd "CHGATR OBJ('$1') ATR(*CCSID) VALUE(819)" exec_cmd "CPYFRMSTMF FROMSTMF('$1') TOMBR('/QSYS.lib/$CUR_LIB.lib/$RPGLE_SRC.file/$member.mbr') MBROPT(*REPLACE)" exec_cmd "CHGPFM FILE($CUR_LIB/$RPGLE_SRC) MBR($member) SRCTYPE(RPGLE) TEXT('$APPLICATION')" exec_cmd "CRTBNDRPG PGM($CUR_LIB/$member) DFTACTGRP(*NO) DBGVIEW(*SOURCE)" $member } # Copy SQLRPGLE to SRCPF and compile crt_sqlrpgle(){ echo -e '\n... executing commands... ' filename=$(basename "$1") member="${filename%.*}" exec_cmd "CHGATR OBJ('$1') ATR(*CCSID) VALUE(819)" exec_cmd "CPYFRMSTMF FROMSTMF('$1') TOMBR('/QSYS.lib/$CUR_LIB.lib/$RPGLE_SRC.file/$member.mbr') MBROPT(*REPLACE)" exec_cmd "CHGPFM FILE($CUR_LIB/$RPGLE_SRC) MBR($member) SRCTYPE(SQLRPGLE) TEXT('$APPLICATION')" exec_cmd "CRTSQLRPGI OBJ($CUR_LIB/$member) SRCFILE($CUR_LIB/$RPGLE_SRC) COMMIT(*NONE) DBGVIEW(*SOURCE)" $member } # Copy CLLE to SRCPF and compile crt_clle(){ echo -e '\n... executing commands... ' filename=$(basename "$1") member="${filename%.*}" exec_cmd "CHGATR OBJ('$1') ATR(*CCSID) VALUE(819)" exec_cmd "CPYFRMSTMF FROMSTMF('$1') TOMBR('/QSYS.lib/$CUR_LIB.lib/$CL_SRC.file/$member.mbr') MBROPT(*REPLACE)" exec_cmd "CHGPFM FILE($CUR_LIB/$CL_SRC) MBR($member) SRCTYPE(CLLE) TEXT('$APPLICATION')" exec_cmd "CRTBNDCL PGM($CUR_LIB/$member) SRCFILE($CUR_LIB/$CL_SRC) DFTACTGRP(*NO) ACTGRP(*CALLER) DBGVIEW(*SOURCE)" $member } search_dspf(){ for FILE in "$IFS_DIR"/* do ext="${FILE##*.}" if [[ $ext == 'dspf' ]]; then echo -e "\n=========== Building DSPF - $FILE =============" crt_dspf $FILE fi done } search_rpgle(){ for FILE in "$IFS_DIR"/* do ext="${FILE##*.}" if [[ $ext == 'rpgle' ]]; then echo -e "\n=========== Building RPGLE - $FILE =============" crt_rpgle $FILE fi done } search_sqlrpgle(){ for FILE in "$IFS_DIR"/* do ext="${FILE##*.}" if [[ $ext == 'sqlrpgle' ]]; then echo -e "\n=========== Building SQLRPGLE - $FILE =============" crt_sqlrpgle $FILE fi done } search_clle(){ for FILE in "$IFS_DIR"/* do ext="${FILE##*.}" if [[ $ext == 'clle' ]]; then echo -e "\n=========== Building CLLE - $FILE =============" crt_clle $FILE fi done } echo -e "|=== Starting to build the programs for $APPLICATION ===|" search_dspf search_rpgle search_sqlrpgle search_clle echo -e '|=================================================================================|' echo -e '| Program build completed, please check if you encountered any errors. |' echo -e '| Check the log files created in the below folder. |' echo -e "| $IFS_DIR |" echo -e '|=================================================================================|' Synchronizing your set up It is necessary that, when you have completed all your customization for VS Code, you should sync your setup so that, if you change your laptop/desktop, those are retained and you don’t have to do it all over again. When you go to File → Preferences →Setting Sync you can configure where you want to sync it. I have currently set it to sync with my GitHub account. Wrapping up… I find the VS Code editor highly customizable and efficient to use while I work on IBM i. It is also available on all operating systems. I hope this article has given you some ideas on how to use the editor. As always, request you to get in touch with me on LinkedIn/Twitter or email with your valuable feedback. Thanks for reading!NodeJS - Consume REST API on IBM i (AS400)2020-10-11T00:00:00-05:002020-10-11T00:00:00-05:00https://www.anandk.dev/2020/10/NodeJS-Webservice-IBMi-AS400<p style="text-align: justify;"><img src="https://www.anandk.dev/assets/images/forposts/currency_header.png" alt="Currency-Query" class="align-center" /><br />
In my last <a href="https://www.anandk.dev/2020/08/Webservice-IBMi-AS400.html">article</a>, we saw how to consume a RESTFul API using native SQL functions. This article gives another method for it by using NodeJS on the IBM i. </p>
<p>NodeJS has been available on IBM i from 2009, yes, it’s that old!</p>
<blockquote>
<p>(Correction! Thanks to <a href="https://twitter.com/kadler_ibm">Kevin Adler</a> from IBM, “while NodeJS was created in 2009, it has only been available on IBM i since 2014.” )</p>
</blockquote>
<p>I assume you know JavaScript basics, but still have provided enough details for beginners<strong>.</strong></p>
<p>Let’s see the approach, the structure, and the tools needed to write such an application.</p>
<h2 id="the-api-we-will-be-consuming">The API we will be consuming</h2>
<h3 id="open-exchange-rates">Open Exchange Rates</h3>
<p style="text-align: justify;">Many business applications need currency rates regularly if the clients they are serving are from various countries. I have chosen this API because it is easy to understand and is easy to update in DB2.</p>
<p style="text-align: justify;">First, we have to sign-up for the API by visiting this <a href="https://openexchangerates.org/signup/free">URL</a>. The free plan allows 1000 requests per month for a default base (‘USD’) currency. The rates are refreshed every 30 minutes. This should be good enough for our application. Please signup and get your API-Key which will be needed in the application.</p>
<p style="text-align: justify;">This API returns currency rates in JSON format, like below (please refer my <a href="https://www.anandk.dev/2020/08/Webservice-IBMi-AS400.html">earlier</a> article for a brief background on JSON).</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"disclaimer"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Usage subject to terms: https://openexchangerates.org/terms"</span><span class="p">,</span><span class="w">
</span><span class="nl">"license"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://openexchangerates.org/license"</span><span class="p">,</span><span class="w">
</span><span class="nl">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="mi">1600678800</span><span class="p">,</span><span class="w">
</span><span class="nl">"base"</span><span class="p">:</span><span class="w"> </span><span class="s2">"USD"</span><span class="p">,</span><span class="w">
</span><span class="nl">"rates"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"AED"</span><span class="p">:</span><span class="w"> </span><span class="mf">3.672951</span><span class="p">,</span><span class="w">
</span><span class="nl">"AFN"</span><span class="p">:</span><span class="w"> </span><span class="mf">76.72753</span><span class="p">,</span><span class="w">
</span><span class="nl">"ALL"</span><span class="p">:</span><span class="w"> </span><span class="mf">104.631252</span><span class="p">,</span><span class="w">
</span><span class="nl">"AMD"</span><span class="p">:</span><span class="w"> </span><span class="mf">481.616228</span><span class="p">,</span><span class="w">
</span><span class="nl">"ANG"</span><span class="p">:</span><span class="w"> </span><span class="mf">1.792053</span><span class="p">,</span><span class="w">
</span><span class="nl">"AOA"</span><span class="p">:</span><span class="w"> </span><span class="mf">618.42</span><span class="p">,</span><span class="w">
</span><span class="nl">"ARS"</span><span class="p">:</span><span class="w"> </span><span class="mf">75.306</span><span class="p">,</span><span class="w">
</span><span class="nl">"AUD"</span><span class="p">:</span><span class="w"> </span><span class="mf">1.373749</span><span class="p">,</span><span class="w">
</span><span class="err">.................</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<h2 id="nodejs-libraries-and-ecosystem">NodeJS libraries and ecosystem</h2>
<p style="text-align: justify;">NodeJS is a powerful engine based on Google’s V8 engine for JavaScript, it is JavaScript outside of a web browser, for server-side programming. It is again enhanced by its vast array of open-source libraries, which are available on a registry <a href="http://www.npmjs.com">www.npmjs.com</a>.</p>
<p style="text-align: justify;">Sometimes, it becomes challenging to decide which library to use for a given function as there are many to choose from. I go with one, which has a large number of weekly downloads and its collaborator is updating/fixing it regularly, so those which have a strong support backbone. This can be checked from its <a href="https://www.npmjs.com/package/needle">npmjs page</a> and <a href="https://github.com/tomas/needle">github page</a>.</p>
<p style="text-align: justify;">These libraries can be used by first installing them in our project directory and then by simply using the <code class="language-plaintext highlighter-rouge">require</code> keyword in JavaScript code, which is similar to how we use copybooks in RPGLE programs.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">needle</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">needle</span><span class="dl">'</span><span class="p">);</span>
</code></pre></div></div>
<p style="text-align: justify;">For this application, we will need 2 libraries, one for connecting to DB2 and the other for getting (GET) HTTP request to consume the API. We will see those two below.</p>
<h3 id="idb-pconnector---promise-based-db2-connector-for-ibm-i"><a href="https://www.npmjs.com/package/idb-pconnector">idb-pconnector - Promise-based DB2 Connector for IBM i</a></h3>
<p style="text-align: justify;">I will be using idb-pconnector but, IBM has recently released its <a href="https://github.com/IBM/ibmi-oss-examples/blob/master/odbc/odbc.md#installation-on-ibm-i">ODBC driver</a> that can be used from Windows, macOS or Linux as well as from IBM i. Going forward, this ODBC library will be used extensively, as ODBC is a standardized API to interact with various DBMS.</p>
<p style="text-align: justify;">At the time of writing this article, I did not have any IBM i server that has this driver installed, hence I will be using idb-pconnector for our DB2 needs. NodeJS has its ODBC library <a href="https://www.npmjs.com/package/odbc">here</a>.</p>
<p style="text-align: justify;">idb-pconnector has various ways that make it easy to connect to the database and run SQL statements or call Stored Procedures. This is a promise-based version of the existing <a href="https://www.npmjs.com/package/idb-connector">idb-connector library</a>. (without <code class="language-plaintext highlighter-rouge">p</code> in connector)</p>
<p style="text-align: justify;">The documentation for this library can be found <a href="https://github.com/IBM/nodejs-idb-pconnector/blob/master/docs/README.md">here</a>.</p>
<p style="text-align: justify;">We will visit briefly on what are promises, async, await functions of JavaScript so that this is easy to understand.</p>
<h3 id="needle---http-client-on-nodejs"><a href="https://www.npmjs.com/package/needle">Needle - HTTP Client on NodeJS</a></h3>
<p style="text-align: justify;">Needle, as per its creator is, “The leanest and most handsome HTTP client in the Nodelands.”</p>
<p style="text-align: justify;">We will be using its GET HTTP request from our API to get the currencies we need in the above-mentioned JSON format.</p>
<h2 id="some-javascript-basics">Some JavaScript Basics</h2>
<h3 id="callbackpromiseasyncawait">Callback/Promise/Async/Await</h3>
<p style="text-align: justify;">I highly recommend that you go through details of this topic <a href="https://javascript.info/async">here</a>. In this article we will just touch upon the concepts to give some idea as to how the code is behaving.</p>
<p style="text-align: justify;">Let’s say when we call multiple programs one by one in a CL program, those are being called <strong>synchronously</strong>, one after the other. The control transfers from one statement to another. On the other hand, when we use the SBMJOB command, the control comes back immediately, giving us the freedom to take the next step while the submitted job is doing its stuff, this is an <strong>asynchronous</strong> way in the JavaScript world!</p>
<p style="text-align: justify;">But if some of our later code is dependent on our submitted job completion, then we have to write extra code to wait for its completion (like checking data queue, or data area), or write code to execute this functionality within the submitted program after completion of its intended function, in JavaScript it is called as callbacks.</p>
<p style="text-align: justify;">Let’s say, we call Program A with its parameter along with one more parameter as the name of program B, which is a callback function. Program A will complete its functionality and will call program B with the appropriate parameters. This way the reusability of Program A increases as it dynamically calls a program for which it gets the name in a parameter. We also get the flexibility to write Program B each time based on our current needs.</p>
<p style="text-align: justify;">There are some challenges with callbacks called as <a href="https://javascript.info/callbacks#pyramid-of-doom">callback hell</a>, hence JavaScript has more advanced methods to do the same called as Promise based Async and Await, which we will be using in our application.</p>
<p style="text-align: justify;">Promise is a special JavaScript object, which gives one of the two outputs, either a result or an error. It is called as Resolved or Rejected.</p>
<p style="text-align: justify;">We are using Async function that returns a promise, which is either resolved (we get the desired result) or rejected (we get an error), based on the outcome, we decide our next step.</p>
<p style="text-align: justify;">Here we use the JavaScript keyword <code class="language-plaintext highlighter-rouge">await</code>, which waits for the promise to be resolved or rejected, and if it is rejected, we write <code class="language-plaintext highlighter-rouge">try</code> and <code class="language-plaintext highlighter-rouge">catch</code> block to monitor for any errors.</p>
<p style="text-align: justify;">In our case with database update, we <code class="language-plaintext highlighter-rouge">await</code> for the execution of our SQL statement; it may go well or may end up in error, which is monitored in a <code class="language-plaintext highlighter-rouge">try</code> / <code class="language-plaintext highlighter-rouge">catch</code> block. The errors are displayed on the screen using <code class="language-plaintext highlighter-rouge">console.log</code> like <code class="language-plaintext highlighter-rouge">DSPLY</code> from where we are executing our application.</p>
<p style="text-align: justify;">As I said, this is a vast topic and cannot be covered in just a few paragraphs. We will concentrate on demonstrating how to build NodeJS application on IBM i, so please revisit the articles on it for which a <a href="https://javascript.info/async">link</a> is given.</p>
<h2 id="how-to-start-writing-nodejs-application">How to start writing NodeJS application?</h2>
<p style="text-align: justify;">We will see some basic steps needed to start writing a NodeJS application. These steps ensure that our application becomes portable and deployable on multiple systems by issuing few commands. <br />
NodeJS comes with its package manager called npm, short for Node Package Manager. This package manager is used to install the libraries as discussed in the previous section, also is a command-line utility to start creating our application.</p>
<p style="text-align: justify;"><strong>Note</strong>: We will be writing this application in IBM i’s PASE environment, so we need to get into it. There are two options, first, call <code class="language-plaintext highlighter-rouge">QP2TERM</code> program from the command line to get to the $ prompt of this environment, second, use an SSH session as described beautifully by Aaron Bartell on <a href="https://www.itjungle.com/2014/09/17/fhg091714-story01/">ITJungle</a>.</p>
<p style="text-align: justify;">I will be using the SSH session, as it is very convenient to issue commands or retrieve earlier issued commands and navigate through the file system.</p>
<h3 id="nodejs-version">NodeJS Version</h3>
<p>I have written and tested this application on PUB400’s IBM i, which is at Version 7.4 for IBM i. NodeJS version can be checked using the below command. Some of the functionality in the application may not work below this version. (QP2TERM SHELL)</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>node <span class="nt">-v</span>
v12.18.2
</code></pre></div></div>
<p>Let’s start with creating our application. Full application with source code is given, still, if you want to do it manually follow the below steps.</p>
<h3 id="1-create-a-directory">1. Create a directory</h3>
<p style="text-align: justify;">We will create a directory in IFS (/home/YOUR_ID/) to store all the sources and downloaded Node libraries. I will call it <code class="language-plaintext highlighter-rouge">currency</code>. Once created, I will change my current directory to <code class="language-plaintext highlighter-rouge">currency</code>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># I am currently in my /home/USERID (home directory)</span>
<span class="nv">$ </span><span class="nb">mkdir </span>currency
<span class="nv">$ </span><span class="nb">cd </span>currency
</code></pre></div></div>
<h3 id="2-initialize-the-application">2. Initialize the application</h3>
<p style="text-align: justify;">When I am in my application directory, I will start initializing the application by using the Node package manager with command <code class="language-plaintext highlighter-rouge">npm init</code>. With this, npm creates two files for us; first, it will create package.json and when we start installing open-source libraries, it will create package-lock.json. These two files are kept updated with the list of installed libraries, their versions, and commands needed by the application.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.
See <span class="sb">`</span>npm <span class="nb">help </span>init<span class="sb">`</span> <span class="k">for </span>definitive documentation on these fields
and exactly what they <span class="k">do</span><span class="nb">.</span>
Use <span class="sb">`</span>npm <span class="nb">install</span> <pkg><span class="sb">`</span> afterwards to <span class="nb">install </span>a package and
save it as a dependency <span class="k">in </span>the package.json file.
Press ^C at any <span class="nb">time </span>to quit.
package name: <span class="o">(</span>currency<span class="o">)</span>
version: <span class="o">(</span>1.0.0<span class="o">)</span>
description: Fetch Currency API
entry point: <span class="o">(</span>index.js<span class="o">)</span> app.js
<span class="nb">test command</span>: node app.js <span class="s1">'USD'</span>
git repository:
keywords:
author: www.anandk.dev
license: <span class="o">(</span>ISC<span class="o">)</span>
About to write to /home/anandk/currency/package.json:
<span class="o">{</span>
<span class="s2">"name"</span>: <span class="s2">"currency"</span>,
<span class="s2">"version"</span>: <span class="s2">"1.0.0"</span>,
<span class="s2">"description"</span>: <span class="s2">"Fetch Currency API"</span>,
<span class="s2">"main"</span>: <span class="s2">"app.js"</span>,
<span class="s2">"scripts"</span>: <span class="o">{</span>
<span class="s2">"test"</span>: <span class="s2">"node app.js 'USD'"</span>
<span class="o">}</span>,
<span class="s2">"author"</span>: <span class="s2">"www.anandk.dev"</span>,
<span class="s2">"license"</span>: <span class="s2">"ISC"</span>
<span class="o">}</span>
Is this OK? <span class="o">(</span><span class="nb">yes</span><span class="o">)</span>
</code></pre></div></div>
<p>Once <code class="language-plaintext highlighter-rouge">npm init</code> command is issued, it starts asking you questions about the application.</p>
<p style="text-align: justify;">I entered application-specific answers like name, description, entry point, test command, author, and kept the rest blank. It is fine if you enter nothing, but would be good to enter the name and description of the application. Finally, it asks if the generated JSON is OK, when confirmed the package.json file is created.</p>
<p style="text-align: justify;">These package.json and package-lock.json files are useful when you want to install our ready application once you download it from <a href="https://github.com/anand-khekale/IBMi">here</a> on the IBM i. After downloading in the appropriate directory you just have to issue command <code class="language-plaintext highlighter-rouge">npm install</code> and all the required dependencies will be installed and the application will be ready to use.</p>
<h3 id="3-installing-the-required-libraries">3. Installing the required libraries</h3>
<p style="text-align: justify;">As stated earlier, we will be using two libraries. We will now install those so that, those can be used in our programs.</p>
<p>First, let’s install our IBM i specific library.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>npm <span class="nb">install </span>idb-pconnector
</code></pre></div></div>
<p>Once this library is installed, we will proceed with installing the HTTP client library Needle</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>npm <span class="nb">install </span>needle
</code></pre></div></div>
<p style="text-align: justify;">After successful installation, I will just verify, if the package.json is updated with dependencies and a package-lock.json file is created.</p>
<h3 id="4-start-writing-our-programs">4. Start writing our programs</h3>
<p style="text-align: justify;">We have everything now to start writing our programs. The basic directory structure I have created is as below. The full source is available on GitHub <a href="https://github.com/anand-khekale/IBMi">here</a>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/home/ANANDK/currency<span class="nv">$ </span>tree
<span class="nb">.</span>
├── app.js
├── db
│ ├── createDb.js
│ └── updateDb.js
├── node_modules
├── package-lock.json
├── package.json
└── utils
├── formatDate.js
└── getCurrency.js
</code></pre></div></div>
<p style="text-align: justify;">Please note, <code class="language-plaintext highlighter-rouge">node_module</code> is the directory created by npm when we start installing the individual libraries; we are not supposed to touch/change that directory.</p>
<p style="text-align: justify;">Let’s go through our sources one by one. I have added enough comments so that additional explanation is not needed.</p>
<h3 id="5-create-table-currency---createdbjs">5. Create table CURRENCY - createDb.js</h3>
<p style="text-align: justify;">Well, we can create the table using DDS, DDL, or using STRSQL, but I just created this source, which will be run for the first time, just to showcase, this can be done from NodeJS as well.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// import required class - DBPoll from idb-pconnector for connection to DB2</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">DBPool</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">idb-pconnector</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// Keep the SQL statements ready</span>
<span class="kd">const</span> <span class="nx">sql</span> <span class="o">=</span> <span class="s2">`CREATE OR REPLACE TABLE APKHEKALE1.CURRENCY(
Date Timestamp(0),
BCURR CHAR(3),
USD Decimal(12, 4),
EUR Decimal(12, 4),
GBP Decimal(12, 4),
INR Decimal(12, 4),
SGD Decimal(12, 4)) RCDFMT RCURRENCY`</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">label</span> <span class="o">=</span> <span class="s2">`LABEL ON COLUMN APKHEKALE1.CURRENCY (
Date TEXT IS 'Date fetched',
BCURR TEXT IS 'Base Currency',
USD TEXT IS 'United States Dollar',
EUR TEXT IS 'Euro',
GBP TEXT IS 'Pound sterling',
INR TEXT IS 'Indian Rupee',
SGD TEXT IS 'Singapore Dollars')`</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">colhdg</span> <span class="o">=</span> <span class="s2">`LABEL ON COLUMN APKHEKALE1.CURRENCY (
Date IS 'Date fetched',
BCURR IS 'Base Currency',
USD IS 'United States Dollar',
EUR IS 'Euro',
GBP IS 'Pound sterling',
INR IS 'Indian Rupee',
SGD IS 'Singapore Dollars')`</span><span class="p">;</span>
<span class="c1">// Write async - promise based function so that await can be used</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">createDB</span><span class="p">()</span> <span class="p">{</span>
<span class="k">try</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">pool</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">DBPool</span><span class="p">();</span> <span class="c1">//Create a new instance of pool</span>
<span class="kd">const</span> <span class="nx">connection</span> <span class="o">=</span> <span class="nx">pool</span><span class="p">.</span><span class="nx">attach</span><span class="p">();</span> <span class="c1">// create the connection</span>
<span class="kd">const</span> <span class="nx">statement</span> <span class="o">=</span> <span class="nx">connection</span><span class="p">.</span><span class="nx">getStatement</span><span class="p">();</span> <span class="c1">//create new instance of statement</span>
<span class="c1">// execute and await each statement, if something goes wrong, we will be in error block. </span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">prepare</span><span class="p">(</span><span class="nx">sql</span><span class="p">);</span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">execute</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">prepare</span><span class="p">(</span><span class="nx">label</span><span class="p">);</span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">execute</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">prepare</span><span class="p">(</span><span class="nx">colhdg</span><span class="p">);</span>
<span class="k">await</span> <span class="nx">statement</span><span class="p">.</span><span class="nx">execute</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">pool</span><span class="p">.</span><span class="nx">detach</span><span class="p">(</span><span class="nx">connection</span><span class="p">);</span> <span class="c1">//Once done, detach/close the connection </span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">//call the function and catch errors, if any.</span>
<span class="nx">createDB</span><span class="p">().</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">})</span>
</code></pre></div></div>
<p style="text-align: justify;">Once the source is ready, while we are in the <code class="language-plaintext highlighter-rouge">currency/db</code> directory, we will issue <code class="language-plaintext highlighter-rouge">node createDb.js</code> command to create the table. (QP2TERM SHELL)</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cd </span>currency/db
<span class="nv">$ </span>node createDb.js
</code></pre></div></div>
<h3 id="6-our-entry-point-of-application---appjs">6. Our entry point of application - app.js</h3>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// import our getCurrency function from utils directory, this is similar to /COPY in RPGLE</span>
<span class="kd">const</span> <span class="nx">getCurrency</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">./utils/getCurrency</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// import our updateDb function from db directory</span>
<span class="kd">const</span> <span class="nx">updateDb</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">./db/updateDb</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">baseCurr</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">];</span> <span class="c1">// accept base currency from command line</span>
<span class="c1">//If base currency is not entered, give error and exit the program.</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">baseCurr</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Base Currency not entered</span><span class="dl">'</span><span class="p">)</span> <span class="p">}</span>
<span class="cm">/* Call getCurrency function passing base currency and a callback function.
The getCurrency will give a call to callback with appropriate parameters,
if it is successful it will have currency data, else it will have error, but not both. */</span>
<span class="nx">getCurrency</span><span class="p">(</span><span class="nx">baseCurr</span><span class="p">,</span> <span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">currencyData</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="c1">// call updateDb for updating the data into DB2 with currency rates. </span>
<span class="nx">updateDb</span><span class="p">(</span><span class="nx">currencyData</span><span class="p">).</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">})</span>
<span class="p">}</span>
<span class="p">})</span>
</code></pre></div></div>
<h3 id="7-get-fetch-the-currency-data---getcurrencyjs">7. Get (fetch) the Currency data - getCurrency.js</h3>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Import Needle library that we installed, to issue HTTP GET request.</span>
<span class="kd">var</span> <span class="nx">needle</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">needle</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// Assign API key that we received from https://openexchangerates.org/signup/free</span>
<span class="kd">const</span> <span class="nx">API_KEY</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">YOUR-KEY-FROM-OPEN-EXCHANGE-RATES</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">getCurrency</span> <span class="o">=</span> <span class="p">(</span><span class="nx">baseCurr</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="s2">`https://openexchangerates.org/api/latest.json?app_id=</span><span class="p">${</span><span class="nx">API_KEY</span><span class="p">}</span><span class="s2">&base=</span><span class="p">${</span><span class="nx">baseCurr</span><span class="p">}</span><span class="s2">`</span>
<span class="nx">needle</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">response</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span> <span class="c1">//Check if there are any communication errors</span>
<span class="nx">callback</span><span class="p">(</span><span class="dl">'</span><span class="s1">Unable to connect to currency services</span><span class="dl">'</span><span class="p">,</span> <span class="kc">undefined</span><span class="p">)</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Check if we received any error from API</span>
<span class="nx">callback</span><span class="p">({</span> <span class="dl">"</span><span class="s2">error</span><span class="dl">"</span><span class="p">:</span> <span class="nx">response</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">description</span> <span class="p">},</span> <span class="kc">undefined</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">error</span> <span class="o">&&</span> <span class="nx">response</span><span class="p">.</span><span class="nx">statusCode</span> <span class="o">==</span> <span class="mi">200</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// All went well and we have the needed data</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">undefined</span><span class="p">,</span> <span class="nx">response</span><span class="p">.</span><span class="nx">body</span><span class="p">);</span> <span class="c1">// Call the callback with no error and data</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="p">}</span>
<span class="cm">/* Since we will be using getCurrency from outside this source, we need to export it, similarly as
we do with RPGLE procedures */</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="nx">getCurrency</span><span class="p">;</span>
</code></pre></div></div>
<h3 id="8-update-the-database---updatedbjs">8. Update the database - updateDb.js</h3>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//import only required class - DBPool from idb-pconnector </span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">DBPool</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">idb-pconnector</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">//get our utility which formats the date for us </span>
<span class="kd">const</span> <span class="nx">formatDate</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">../utils/formatDate</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">//Write async function as we will use promise based await while updating the database</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">updateDb</span><span class="p">(</span><span class="nx">currencyData</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Format the date in IBM i timestamp format as CCYY-MM-DD-HH.MM.SS</span>
<span class="kd">const</span> <span class="nx">timestamp</span> <span class="o">=</span> <span class="nx">formatDate</span><span class="p">(</span><span class="nx">currencyData</span><span class="p">.</span><span class="nx">timestamp</span><span class="p">)</span>
<span class="c1">// Select required currency to be updated into database from the JSON</span>
<span class="kd">const</span> <span class="nx">baseCurr</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">base</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">currUSD</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">rates</span><span class="p">.</span><span class="nx">USD</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">currEUR</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">rates</span><span class="p">.</span><span class="nx">EUR</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">currGBP</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">rates</span><span class="p">.</span><span class="nx">GBP</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">currINR</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">rates</span><span class="p">.</span><span class="nx">INR</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">currSGD</span> <span class="o">=</span> <span class="nx">currencyData</span><span class="p">.</span><span class="nx">rates</span><span class="p">.</span><span class="nx">SGD</span><span class="p">;</span>
<span class="c1">// Create a database connection and update the database. </span>
<span class="k">try</span> <span class="p">{</span>
<span class="c1">// Connect DB2 using a pool of connection. This is useful for scalability.</span>
<span class="kd">const</span> <span class="nx">pool</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">DBPool</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">sql</span> <span class="o">=</span> <span class="s2">`Insert into APKHEKALE1.CURRENCY values (?,?,?,?,?,?,?) WITH NONE`</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">params</span> <span class="o">=</span> <span class="p">[</span><span class="nx">timestamp</span><span class="p">,</span> <span class="nx">baseCurr</span><span class="p">,</span> <span class="nx">currUSD</span><span class="p">,</span> <span class="nx">currEUR</span><span class="p">,</span> <span class="nx">currGBP</span><span class="p">,</span> <span class="nx">currINR</span><span class="p">,</span> <span class="nx">currSGD</span><span class="p">];</span>
<span class="k">await</span> <span class="nx">pool</span><span class="p">.</span><span class="nx">prepareExecute</span><span class="p">(</span><span class="nx">sql</span><span class="p">,</span> <span class="nx">params</span><span class="p">);</span> <span class="c1">// prepare and execute the statement </span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// if something goes wrong, like SQL error, catch those here. </span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">error</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">//Export the module, so that it can be used from outside this source. </span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="nx">updateDb</span><span class="p">;</span>
</code></pre></div></div>
<h3 id="9-running-the-application">9. Running the application</h3>
<p style="text-align: justify;">There are two ways you can call the node application, from your SSH or QP2TERM session, or; from the CL/RPGLE program. We will see it one by one.</p>
<h3 id="a-running-from-pase-environment">a. Running from PASE environment</h3>
<p style="text-align: justify;">From your SSH/QP2TERM session, change your current directory to <code class="language-plaintext highlighter-rouge">currency</code>. Make sure that the PF is created in your library, use the below command to run the application.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>node app.js <span class="s1">'USD'</span>
</code></pre></div></div>
<p>Command run from QP2TERM session (CALL QP2TERM)<br />
<img src="https://www.anandk.dev/assets/images/forposts/currency_qp2term.png" alt="QP2TERM-View" class="align-center" /></p>
<p style="text-align: justify;">Here we are using node to run our first entry point, that is app.js, and passing ‘USD’ as the base currency. Once the application runs fine, we will be back to our $ prompt.</p>
<p>We can also utilize the package.json file which got created when we started building our application, we provided our <code class="language-plaintext highlighter-rouge">test</code> script to be <code class="language-plaintext highlighter-rouge">node app.js 'USD'</code> . Enter the command as shown below (QP2TERM SHELL).</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cd </span>currency
<span class="nv">$ </span>npm run <span class="nb">test</span>
<span class="o">></span> currency@1.0.0 <span class="nb">test</span> /home/ANANDK/currency
<span class="o">></span> node app.js <span class="s1">'USD'</span>
<span class="err">$</span>
</code></pre></div></div>
<p style="text-align: justify;">We can check if our CURRENCY table got updated to make sure everything is working as expected (I ran this multiple times on different days, hence these many entries).</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/currency_header.png" alt="Currency-Query" class="align-center" /></p>
<p>We can test our validation for a base currency, without passing it as a parameter, see below.
<img src="https://www.anandk.dev/assets/images/forposts/currency_validation.png" alt="Base-Curr-Validation" class="align-center" /></p>
<h3 id="b-running-from-cl-programs">b. Running from CL programs</h3>
<p style="text-align: justify;">To run any program in the PASE environment, I feel inclined to writing a small shell script to take care of navigating to the correct directory. The script is as below named as runcurrency.sh.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/QOpenSys/usr/bin/sh</span>
<span class="c"># This line with '#' is a comment, the above line with '#!' is a Shebang, by which the shell knows which shell to use.</span>
<span class="c"># We have used 'sh' shell on the IBM i which is mostly available by default, alternatively one may use </span>
<span class="c"># /QOpenSys/pkgs/bin/bash (to check if it's available, do a cd /QOpenSys/pkgs/bin/ if there are no errors, bash is </span>
<span class="c"># available on your IBM i)</span>
<span class="c"># For more information visit - https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzalf/rzalfpase.htm</span>
<span class="c"># The below line runs our application by navigating to the app.js in the directory structure. </span>
node /home/ANANDK/currency/app.js <span class="s1">'USD'</span>
</code></pre></div></div>
<p style="text-align: justify;">Whenever a shell script is written in a file, the file needs to be converted into an executable, this is done by issuing the below command (from QP2TERM shell prompt).</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cd </span>currency
<span class="nv">$ </span><span class="nb">chmod</span> +x runcurrency.sh
</code></pre></div></div>
<p>Then it becomes easy to call this shell script from a CL program.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PGM
QSH CMD<span class="o">(</span><span class="s1">'/home/ANANDK/currency/runcurrency.sh'</span><span class="o">)</span>
ENDPGM
</code></pre></div></div>
<h3 id="10-installing-the-application">10. Installing the application</h3>
<p style="text-align: justify;">You can directly install this application by downloading the folder from GitHub <a href="https://github.com/anand-khekale/IBMi">here</a>. Once the folder is downloaded follow the below steps.</p>
<ol>
<li>From a text editor change the below sources.
<ul>
<li>Change the API Key in getCurrency.js - Replace ‘YOUR-KEY-FROM-OPEN-EXCHANGE-RATES’ with the key you obtained from the API provider.</li>
<li>Change library in createDb.js</li>
<li>Change library in updateDb.js</li>
<li>Change path with your ID, in runcurrency.sh ==> /home/<strong>YourID</strong>/currency/app.js</li>
<li>Change path with your ID in RUNNODE.CLLE ==> /home/<strong>YourID</strong>/currency/runcurrency.sh</li>
</ul>
</li>
<li>
<p>After updating the sources, upload the folder <code class="language-plaintext highlighter-rouge">currency</code> into your IFS <code class="language-plaintext highlighter-rouge">/home/YourID/</code> directory. IBM’s Access Client Solution has <code class="language-plaintext highlighter-rouge">Actions→Integrated File System→Actions→Upload</code> to upload the files.</p>
<p>I use <a href="https://code.visualstudio.com/">VSCODE</a> with <a href="https://marketplace.visualstudio.com/items?itemName=Kelvin.vscode-sshfs">SSHFS</a> extension, and write code directly into IFS on the IBM i from my computer. I will write an article explaining how to do this.</p>
</li>
<li>Install node libraries
<ul style="text-align: justify;">
<li>Call QP2TERM, on $ prompt execute the below one after the other (package.json and package-lock.json files are necessary for this step). This step will take some time and will provide details of libraries installed.</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span><span class="nb">cd </span>currency
<span class="nv">$ </span>npm <span class="nb">install</span>
</code></pre></div> </div>
</li>
<li>
<p>Make the <code class="language-plaintext highlighter-rouge">runcurrency.sh</code> as executable from QP2TERM shell, while you are in <code class="language-plaintext highlighter-rouge">currency</code> directory.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span><span class="nb">chmod</span> +x runncurency.sh
</code></pre></div> </div>
</li>
<li>
<p>Create our database table (PF) running createDb.js (QP2TERM shell).</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span><span class="nb">cd </span>currency/db
<span class="nv">$ </span>node createDb.js
<span class="err">$</span>
</code></pre></div> </div>
</li>
<li>
<p>Create RUNNODE CLLE program (From CL CMD), copy and compile the CL source.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> CPYFRMSTMF FROMSTMF('/home/**YOURID**/currency/RUNNODE.CLLE') TOMBR('/QSYS.lib/**YOURLIB**.lib/QCLSRC.file/RUNNODE.mbr') MBROPT(*REPLACE)
</code></pre></div> </div>
<p>Run the application as per instructions given in earlier section.</p>
</li>
</ol>
<h2 id="conclusion">Conclusion</h2>
<p style="text-align: justify;">I hope this article gave you some ideas on how to create and use NodeJS to consume a RESTful API. The development time is considerably lower, as NodeJS has various libraries at our disposal to quickly write any application.</p>
<p style="text-align: justify;">Special thanks to <a href="http://www.pub400.com">PUB400</a> for hosting free IBM i with latest stuff on it.</p>
<p style="text-align: justify;">As always, request you to get in touch with me on <a href="https://www.linkedin.com/in/anandkdev">LinkedIn</a>/<a href="https://twitter.com/anandkdev">Twitter</a> or <a href="mailto:anand.khekale@gmail.com">email</a> with your valuable feedback. Thanks for reading!</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn my last article, we saw how to consume a RESTFul API using native SQL functions. This article gives another method for it by using NodeJS on the IBM i. NodeJS has been available on IBM i from 2009, yes, it’s that old! (Correction! Thanks to Kevin Adler from IBM, “while NodeJS was created in 2009, it has only been available on IBM i since 2014.” )Consume a REST Web service on IBM i (AS400) using SQL2020-08-31T00:00:00-05:002020-08-31T00:00:00-05:00https://www.anandk.dev/2020/08/Webservice-IBMi-AS400<p style="text-align: justify;"><img src="https://www.anandk.dev/assets/images/forposts/Weather_screen.png" alt="image-center" class="align-center" /> <br />
In this post, we will see how we can build a weather app natively on IBM i using SQL, RPGLE and HTTPGETCLOB, JSON_TABLE functions. The app will consume RESTful APIs to get the data in JSON format. Db2 for i has both the SQL functions available from V7.2 onwards; this functionality will work if you are on or above that version of the OS. </p>
<h3 id="first-some-background-about-json">First, some background about JSON.</h3>
<p style="text-align: justify;">JSON is JavaScript Object Notation, a format in which data is exchanged in “Key”:”Value” pair enclosed in curly braces {}.
Below is the simple example of student information in JSON.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nl">"Student1"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Mathematics"</span><span class="p">:</span><span class="w"> </span><span class="mi">80</span><span class="p">,</span><span class="w">
</span><span class="nl">"Physics"</span><span class="p">:</span><span class="w"> </span><span class="mi">95</span><span class="p">,</span><span class="w">
</span><span class="nl">"Computers"</span><span class="p">:</span><span class="w"> </span><span class="mi">85</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p>As can be seen, if we need to get marks in Physics from the JSON, we have to navigate it as <code class="language-plaintext highlighter-rouge">Student1.Physics</code> to access it’s value and we get 95.</p>
<p>There are various ways JSON can be presented, sometimes it contains array of information nested for a key element.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nl">"Name"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"John Doe"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Phones"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="nl">"Office"</span><span class="p">:</span><span class="w"> </span><span class="mi">25252525</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="nl">"Home"</span><span class="p">:</span><span class="w"> </span><span class="mi">35353535</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p style="text-align: justify;">Here, there are 2 entries for Key <code class="language-plaintext highlighter-rouge">"Phones"</code>, hence if we want only Office phone number, we have to navigate with array notation as <code class="language-plaintext highlighter-rouge">Phones[0].Office</code>. The array index starts at zero. We will be using similar JSON in our application, hence this background.</p>
<h3 id="second-web-services-we-will-be-using">Second, Web Services we will be using.</h3>
<p>To get started, we will start by using two web services for our application.</p>
<ol>
<li><strong>Mapbox Web service</strong> <a href="https://account.mapbox.com/auth/signup/">Link</a>:<br />
The Mapbox Web service accepts City/Area as input and provides Latitude, Longitude and details about the area in a JSON format. <a href="https://raw.githubusercontent.com/anand-khekale/IBMi/master/WeatherApp/mapbox.json">See the JSON Response</a></li>
<li><strong>Weatherstack Web service</strong> <a href="https://weatherstack.com/signup/free">Link</a>:<br />
The Weatherstack API needs Latitude, Longitude for it to give weather forecast. This also gives JSON output to consume. <a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/weatherstack.json">See the JSON Response</a></li>
</ol>
<p>These APIs are free to sign-up. Once signed-up, we need to register/create an app on both APIs and get API Key or Token, also the URL. This API Key will be needed for our request to be identified and processed. We will be using the API keys provided by respective APIs while processing our requests.</p>
<h3 id="third-the-building-blocks-of-our-application">Third, the building blocks of our application.</h3>
<p><strong>DB2 for i</strong> has various HTTP functions available in library <code class="language-plaintext highlighter-rouge">SYSTOOLS</code>. We will be using mainly below 3 functions.</p>
<ul>
<li>
<p><strong>HTTPGETCLOB</strong><br />
This function accepts the URL of the web service and sends a <code class="language-plaintext highlighter-rouge">HTTP GET</code> request. If the request is successful, it gives back the default response provided by the web service provider. It may be in XML or JSON or any other HTTP supported format.<br />
HTTPGETCLOB is used as below:<br />
<code class="language-plaintext highlighter-rouge">HTTPGETCLOB(<Web URL>, <HTTP Header>)</code></p>
</li>
<li>
<p><strong>URLENCODE</strong><br />
This function is used to construct the URL needed by HTTPGETCLOB. The encoding of the URL needs to be correct for the web service to process it. Some characters are escaped while sending the request (e.g. <code class="language-plaintext highlighter-rouge">space</code> is replaced by either <code class="language-plaintext highlighter-rouge">+</code> sign or <code class="language-plaintext highlighter-rouge">%20</code>. e.g. Los Angeles becomes Los%20Angeles). URLENCODE comes handy for this and is highly recommended.<br />
URLENCODE is used as below:<br />
<code class="language-plaintext highlighter-rouge">URLENCODE(<Value to be encoded>, <Encoding format, e.g.UTF-8 which is default>)</code></p>
</li>
<li>
<p><strong>JSON_TABLE</strong><br />
JSON_TABLE parses the JSON that is returned by the HTTPGETCLOB function. Using this function, we can extract the fields we are interested in from the response.<br />
JSON_TABLE is used as below:<br />
<code class="language-plaintext highlighter-rouge">Select * from JSON_TABLE(<JSON-context-item>, '$' COLUMNS (Column-Name Data-type PATH column-path-expression)) AS X</code></p>
</li>
</ul>
<h3 id="fourth-building-our-sql-procedures">Fourth, building our SQL procedures.</h3>
<p>We will be creating two SQL procedures for our two APIs.</p>
<ul>
<li><em>getGeoCode</em></li>
<li><em>getForecast</em></li>
</ul>
<p>SQL procedures can be created using <code class="language-plaintext highlighter-rouge">RUNSQLSTM</code> or using <em>Run SQL Script</em> option of IBM’s Access Client Solution.<br />
We will directly jump into the code, so that, it is clear.</p>
<p class="notice--warning"><strong>Watch out!</strong> <code class="language-plaintext highlighter-rouge">RUNSQLSTM</code> has given me issues with EBCDIC conversion of <code class="language-plaintext highlighter-rouge">$</code> and <code class="language-plaintext highlighter-rouge">[ ]</code>. At the moment, I don’t seem to have solution. I have tried changing the job CCSID to 37, still, it didn’t work. Using ACS’s Run SQL Script is encouraged to create the procedures.</p>
<p>The first SQL procedure is to fetch the location coordinates for the city passed, the code goes as below,</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
</pre></td><td class="code"><pre><span class="k">CREATE</span> <span class="k">OR</span> <span class="k">REPLACE</span> <span class="k">PROCEDURE</span> <span class="n">YourLibrary</span><span class="p">.</span><span class="n">getGeoCode</span><span class="p">(</span>
<span class="k">IN</span> <span class="n">City</span> <span class="nb">CHAR</span><span class="p">(</span><span class="mi">50</span><span class="p">),</span>
<span class="k">IN</span> <span class="n">APIKEY</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">200</span><span class="p">),</span>
<span class="k">OUT</span> <span class="n">Latitude</span> <span class="nb">FLOAT</span><span class="p">,</span>
<span class="k">OUT</span> <span class="n">Longitude</span> <span class="nb">FLOAT</span><span class="p">,</span>
<span class="k">OUT</span> <span class="n">Place</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">LANGUAGE</span> <span class="k">SQL</span>
<span class="k">RESULT</span> <span class="k">SETS</span> <span class="mi">0</span>
<span class="k">BEGIN</span>
<span class="k">SELECT</span> <span class="n">Lat</span><span class="p">,</span> <span class="n">Lon</span><span class="p">,</span> <span class="n">PName</span>
<span class="k">INTO</span> <span class="n">Latitude</span><span class="p">,</span> <span class="n">Longitude</span><span class="p">,</span> <span class="n">Place</span>
<span class="k">FROM</span> <span class="n">JSON_TABLE</span><span class="p">(</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">HTTPGETCLOB</span><span class="p">(</span>
<span class="s1">'https://api.mapbox.com/geocoding/v5/mapbox.places/'</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="k">TRIM</span><span class="p">(</span><span class="n">City</span><span class="p">),</span> <span class="s1">''</span><span class="p">)</span> <span class="n">CONCAT</span>
<span class="s1">'.json?access_token='</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="k">TRIM</span><span class="p">(</span><span class="n">APIKEY</span><span class="p">),</span><span class="s1">''</span><span class="p">)</span> <span class="n">CONCAT</span>
<span class="s1">'&limit='</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="s1">'1'</span><span class="p">,</span><span class="s1">''</span><span class="p">),</span> <span class="k">NULL</span>
<span class="p">),</span>
<span class="s1">'$'</span>
<span class="n">COLUMNS</span>
<span class="p">(</span><span class="n">Lat</span> <span class="nb">FLOAT</span> <span class="n">path</span> <span class="s1">'$.features[0].center[1]'</span><span class="p">,</span>
<span class="n">Lon</span> <span class="nb">FLOAT</span> <span class="n">path</span> <span class="s1">'$.features[0].center[0]'</span><span class="p">,</span>
<span class="n">PName</span> <span class="nb">CHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="n">path</span> <span class="s1">'$.features[0].place_name'</span>
<span class="p">)</span> <span class="n">error</span> <span class="k">on</span> <span class="n">error</span>
<span class="p">)</span> <span class="k">as</span> <span class="n">x</span><span class="p">;</span>
<span class="k">END</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Here, we will see how the code is functioning,</p>
<table>
<thead>
<tr>
<th>Line</th>
<th style="text-align: left">Explanation</th>
</tr>
</thead>
<tbody>
<tr>
<td>1-9</td>
<td style="text-align: left">We are using standard SQL create procedure with all the input and output parameters we need.</td>
</tr>
<tr>
<td>12-14</td>
<td style="text-align: left">To use JSON_TABLE, we have to use select statement with the fields we are interested in. We are selecting Latitude, Longitude and Place details.</td>
</tr>
<tr>
<td>15-22</td>
<td style="text-align: left">HTTPGETCLOB expects web-url and optional HTTP header as it’s parameters, such as <code class="language-plaintext highlighter-rouge">SYSTOOLS.HTTPGETCLOB(URL,null)</code>, here we are not sending any header related information in our request, hence we have set it to NULL. The URL is constructed using URLENCODE as described above.</td>
</tr>
<tr>
<td>23</td>
<td style="text-align: left">We are telling JSON_TABLE from where to extract information from the JSON. <code class="language-plaintext highlighter-rouge">$</code> means we want it from the root of it i.e. from first curly brace.</td>
</tr>
<tr>
<td>24-27</td>
<td style="text-align: left">We select the columns we are interested in, declare the data type and provide the path to the field. This is similar to what we saw above about how to navigate a JSON while extracting office phone number.</td>
</tr>
<tr>
<td>28</td>
<td style="text-align: left">We use <code class="language-plaintext highlighter-rouge">Error on error</code> so that, if there are any issues processing the request, those are reported.</td>
</tr>
</tbody>
</table>
<p>Our second procedure is similar with the required input/output fields and the web service URL.</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="code"><pre><span class="k">CREATE</span> <span class="k">OR</span> <span class="k">REPLACE</span> <span class="k">PROCEDURE</span> <span class="n">YourLibrary</span><span class="p">.</span><span class="n">getForecast</span><span class="p">(</span>
<span class="k">IN</span> <span class="n">Latitude</span> <span class="nb">FLOAT</span><span class="p">,</span>
<span class="k">IN</span> <span class="n">Longitude</span> <span class="nb">FLOAT</span><span class="p">,</span>
<span class="k">IN</span> <span class="n">APIKEY</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">200</span><span class="p">),</span>
<span class="k">OUT</span> <span class="n">W_Description</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">500</span><span class="p">),</span>
<span class="k">OUT</span> <span class="n">CurrentTemp</span> <span class="nb">FLOAT</span>
<span class="p">)</span>
<span class="k">LANGUAGE</span> <span class="k">sql</span>
<span class="k">RESULT</span> <span class="k">SETS</span> <span class="mi">0</span>
<span class="k">BEGIN</span>
<span class="k">SELECT</span> <span class="n">W_Desc</span><span class="p">,</span> <span class="k">Temp</span>
<span class="k">INTO</span> <span class="n">W_Description</span><span class="p">,</span> <span class="n">CurrentTemp</span>
<span class="k">FROM</span> <span class="n">JSON_TABLE</span><span class="p">(</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">HTTPGETCLOB</span><span class="p">(</span>
<span class="s1">'http://api.weatherstack.com/current?access_key='</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="k">TRIM</span><span class="p">(</span><span class="n">APIKEY</span><span class="p">),</span> <span class="s1">''</span><span class="p">)</span> <span class="n">CONCAT</span>
<span class="s1">'&query='</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="n">Latitude</span><span class="p">,</span><span class="s1">''</span><span class="p">)</span> <span class="n">CONCAT</span>
<span class="s1">','</span> <span class="n">CONCAT</span>
<span class="n">SYSTOOLS</span><span class="p">.</span><span class="n">URLENCODE</span><span class="p">(</span><span class="n">Longitude</span><span class="p">,</span><span class="s1">''</span><span class="p">),</span> <span class="k">NULL</span>
<span class="p">),</span>
<span class="s1">'$'</span>
<span class="n">COLUMNS</span>
<span class="p">(</span><span class="n">W_Desc</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">500</span><span class="p">)</span> <span class="n">path</span> <span class="s1">'$.current.weather_descriptions[0]'</span><span class="p">,</span>
<span class="k">Temp</span> <span class="nb">FLOAT</span> <span class="n">path</span> <span class="s1">'$.current.temperature'</span>
<span class="p">)</span> <span class="n">error</span> <span class="k">on</span> <span class="n">error</span>
<span class="p">)</span> <span class="k">as</span> <span class="n">x</span><span class="p">;</span>
<span class="k">END</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="fifth-writing-sqlrpgle-to-use-sql-procedures">Fifth, writing SQLRPGLE to use SQL procedures.</h3>
<p>We will be writing simple DSPF and SQLRPGLE to demonstrate the functionality. I will not post the whole RPGLE program, but will just give a snapshot of main subroutine to get the idea. The source code is hosted at GitHub and links are given in the below section.</p>
<p><img src="https://www.anandk.dev/assets/images/forposts/sqlrpg.png" alt="image-center" class="align-center" /></p>
<p>Our application is ready to consume the web service! We can add error processing, capturing HTTP response status codes, etc. I kept the scope limited, so as to demonstrate how everything fits into the whole picture.</p>
<h3 id="update-7th-sept-some-observations-and-feedback-received-on-this-article">UPDATE (7th Sept.): Some observations and feedback received on this article.</h3>
<p style="text-align: justify;">Thanks to the feedback received on this article, it is noted that, for some reason <code class="language-plaintext highlighter-rouge">SYSTOOLS.HTTPGETCLOB</code> gives <code class="language-plaintext highlighter-rouge">java.net.SocketException</code> for the first time the program/procedure is run. If this error is ignored and the program retried, it seems to work afterwards. Any suggestions/feedback is welcome on this issue.
This is tested on IBM i V 7.4.</p>
<h4 id="source-codes">Source codes</h4>
<ol>
<li><a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/weather.sqlrpgle">Weather.sqlrpgle</a></li>
<li><a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/WEATHERDS.dspf">Weather DSPF</a></li>
<li><a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/getGeoCode.sql">getGeoCode.sql</a></li>
<li><a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/getForecast.sql">getForecast.sql</a></li>
<li><a href="https://raw.githubusercontent.com/anand-khekale/IBMi/master/WeatherApp/mapbox.json">MapBox JSON Response</a></li>
<li><a href="https://github.com/anand-khekale/IBMi/blob/master/WeatherApp/weatherstack.json">Weatherstack JSON Response</a></li>
</ol>
<h4 id="additional-reading">Additional reading</h4>
<ul>
<li><a href="https://developer.ibm.com/articles/i-json-table-trs/">JSON_TABLE</a></li>
<li><a href="https://developer.ibm.com/tutorials/dm-1105httprestdb2/">HTTPGETCLOB</a></li>
<li><a href="https://www.ibm.com/support/knowledgecenter/en/SSEPEK_12.0.0/sqlref/src/tpc/db2z_udf_urlencode.html">URLENCODE</a></li>
</ul>
<p>Feel free to reach me on <a href="https://twitter.com/anandkhekale">Twitter</a>.</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devIn this post, we will see how we can build a weather app natively on IBM i using SQL, RPGLE and HTTPGETCLOB, JSON_TABLE functions. The app will consume RESTful APIs to get the data in JSON format. Db2 for i has both the SQL functions available from V7.2 onwards; this functionality will work if you are on or above that version of the OS. How to connect PostgreSQL from IBM i (AS/400)2020-08-13T00:00:00-05:002020-08-13T00:00:00-05:00https://www.anandk.dev/2020/08/PostgreSQL-IBMi-AS400<p>Here I explain how to connect PostgreSQL database from IBM i using type 4 JDBC driver in a RPGLE program.</p>
<p>Recently, my client acquired an application which uses PostgreSQL database. The main application resides on IBM i. They wanted to get the data from PostgreSQL into IBM i.</p>
<p>I did check, if they are open to install PostgreSQL on IBM i itself, which may save them additional infrastructure costs, but the “management” was not ready and I had no power/influence on their decision.</p>
<p>Long story short, they asked me to confirm if we can connect in any way.</p>
<p>I started on the task, downloaded <a href="https://www.scottklement.com/jdbc/">Scott Klement’s</a> JDBC driver Service Program.
Here are the steps I took,</p>
<ol>
<li>Downloaded type 4 JDBC driver jar from <a href="https://jdbc.postgresql.org">PostgreSQL</a> server, placed it in home folder in IFS. e.g. <code class="language-plaintext highlighter-rouge">/home/UserID/java/jdbc/POSTGRESQL-42.2.14.jar</code></li>
<li>Derived their Class.forName string <code class="language-plaintext highlighter-rouge">'org.postgresql.Driver'</code> from their <a href="https://jdbc.postgresql.org/documentation/documentation.html">documentation</a>. This is needed when connecting to the database, see the source.</li>
<li>There are few environment variables which needs to be set for Java to work from RPGLE.
<ul>
<li>the environment variable - <code class="language-plaintext highlighter-rouge">CLASSPATH</code> for driver <strong>.jar</strong> file in <em>IFS</em>.</li>
<li>setting <code class="language-plaintext highlighter-rouge">STDIN</code>, <code class="language-plaintext highlighter-rouge">STDOUT</code>, <code class="language-plaintext highlighter-rouge">STDERR</code> to catch Java exceptions. Also, a call to <code class="language-plaintext highlighter-rouge">CHECKSTDIO</code> program to set these 3 open. <a href="https://www.ibm.com/developerworks/rational/cafe/docBodyAttachments/2681-102-2-7220/Troubleshooting_RPG_Calls_To_Java_v2.html"><em>more details here</em></a>.</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">RPGLE</code> program source is <a href="https://github.com/anand-khekale/PostgreSQL-IBMi">here</a>.</li>
<li>With RPGLE, Java and correct JDBC driver, this works perfectly.</li>
</ol>
<p>Thanks to the <code class="language-plaintext highlighter-rouge">JDBCR4</code> service program provided by Scott Klement, the development time has shortened considerably.
Feel free to reach me on <a href="https://twitter.com/anandkhekale">Twitter</a>.</p>Anand Khekaleanand.khekale@gmail.comhttps://www.anandk.devHere I explain how to connect PostgreSQL database from IBM i using type 4 JDBC driver in a RPGLE program.