HTML forms & CGI C-Shell scripting

HTML forms are used to select different kinds of user input (Here's a quick tutorial). For example, open your favourite text editor, copy-paste the commands from listing 18.6. HTML -- Form processing example from here) and save the file as text.html.

Then from your browser, open the file to see what the form looks like. It should look like this:


Reading the form data from your shell script is a bit more tricky. You will probably have encountered html pages that look like http://www.someaddress.com/somescript.cgi?param1=value1&param2=value2

value1 and value2 are the parameters that your HTML form has submitted when you hit the submit button (assuming your form only has 2 parameters).

To read them in your c-shell cgi script, use
set input = $<>param1=value1&param2=value2)

so then you will have to split the string and read the parameters individually in different variables:
set param1 = `echo $input | cut -d '&' -f1 | cut -d '=' -f2`
and the same for param2.

Editing multiple files with sed

Sed is a very useful stream editor for Linux.
A stream editor is used to perform basic text transformations
 on an input stream, such as replacing a text string with another
 text string. While in some ways similar to an editor which permits
 scripted edits(such as ed), sed works by making only one
pass over the input(s), and is consequently more efficient. But it
 is sed's ability to filter text in a pipeline which particularly
 distinguishes it from other  types of editors.

For example, if you want to replace the text "a rainy day" with
the text "a bright and starry night" in a text file containing
the sentence "It was a rainy day outside.", you can do it from the prompt
like this:

sed -i "s/a rainy day/a bright and starry night/g" textfile.txt

This command will have changed the text in your file to "It was a bright
and starry night outside.". The nice thing is that you can use it to edit
the same string in multiple files at once. You can also use other
separators than "/". The above command would have been equally valid
 if it was written as:

sed -i "s|a rainy day|a bright and starry night|g" textfile.txt

All in all, sed is quite a powerful way to manipulate strings.

Timing program executions on Linux

Linux has a useful command to do this: time

The time command runs the specified program command with the given arguments. When command finishes, time writes a message to standard output giving timing statistics about this program run. These statistics consist of (i) the elapsed real time between invocation and termination, (ii) the user CPU time (the sum of the tms_utime and tms_cutime values in a struct tms as returned by times(2)), and (iii) the system CPU time (the sum of the tms_stime and tms_cstime values in a struct tms as returned by times(2)).

For example, if your c-shell script is called blah.csh and takes two arguments (say, arg_1 and arg_2), you can find out how long it takes to run by executing it as:
> time source blah.csh arg_1 arg_2

Creating simple plots with Python

Making plots with Python is easy provided you've downloaded the matplotlib plotting library. You will also probably need Numpy for the numerical routines it provides.

The following code example will produce two interactive figures. The first one demonstrates how to overplot curves, the second one shows how to get two separate plots on the figure.

#First, import the libraries as plt and np
import matplotlib.pyplot as plt
import numpy as np

#Define a function
def f(t):
return np.exp(-t)*np.cos(2*np.pi*t)

# Set up two arrays
t = np.arange(0.0, 5.0, 0.1)
t2 = np.arange(0.0, 5.0, 0.02)

#make single plot with overplots
plt.plot(t, f(t), 'g^', t, f(t*t), 'r--', t, f(t*t*t), 'bs')
plt.plot(t2, f(t2), 'g', t2, f(t2*t2), 'r', t2, f(t2*t2*t2), 'b')

# add labels
plt.ylabel('y values')
plt.xlabel('x values')

# Example of how to get math characters.
# Not that the format is the same as TeX markup but you don't need to have
# TeX installed since matplotlib has it's own parser, layout engine and fonts.
# For example, to print the greek letter sigma as the x title, use:
# plt.xlabel(r'$\sigma$')
plt.title(r'$\Delta\chi^2$')

# add some text on the plot at location 2, 0.6
plt.text(2, 0.6, r'$\alpha_i=100,\ \Delta\chi^2=15$')

# use gridmarks
plt.grid(True)

#make second plot with 2 subplots
plt.figure(2)

#first subplot
plt.subplot(211)
plt.plot(t, f(t), 'bo', t2, f(t2), 'k')
plt.ylabel('y values')
plt.xlabel('x values')
plt.title('blah')
plt.grid(True)

#second subplot
plt.subplot(212)
plt.plot(t2,np.cos(2*np.pi*t2), 'r--')
plt.ylabel('y2 Values')
plt.xlabel('x2 values')
plt.grid(True)

# realise plot
plt.show()

This snippet will produce the following figures:

Python vs IDL. The intricacies of the "where" statement.

Selecting specific elements from arrays by means of their index is a quite useful tool when you're manipulating huge data files. In the past I had been using mainly IDL (great but very expensive licensed software) and standard C-shell scripting to do most of the processing but I have recently started to experiment with Python v2.5 (open-source ftw!) and specifically the Enthought and Python(x,y) distributions which, among other things, contain the Matplotlib and SciPy libraries.

Here's how I used to do it in IDL:
First set up a test array called data
IDL>data = findgen(10)
This statement will create an integer array with 10 elements, from 0 to 9.
To select a part of the elements then use:
IDL> data_sub = data(where(data lt 8 and data gt 3))
data_sub now contains the elements 4,5,6,7

In Python we can do something similar using Numpy.
At the Python prompt:
>>> import numpy as np
Set up the test data array as previously. In Python we can do that with:
>>> data = np.arange(0,10,1) # from 0 to 9 incrementing by 1

Now define the limits
>>> lim1 = data > 3
>>> lim2 = data <>>> data_sub = data[lim1 & lim2]
data_sub is now an array with the values 4,5,6,7

It is possible to easily replace specific elements with zero values:
>>> data_zeros = np.where(data > 5, 0, data)
will replace all array elements with a value greater than 5 with 0.
data_zeros is then this array: 0,1,2,3,4,5,0,0,0,0