Other Stuff

Explanation

This chapter is a collection of things that CGI programmers ought to know about, but do not easily fit into other chapters.

Running external programs

One advantage of programming on the server is the ability to utilize programs that run on the server. This is an important technology, and it can also be a serious security risk, so you need to understand how this works. This topic is dependant on the operating system that the server is running. We will concentrate on unix, because it is frequently used for CGI work, and because many programmers are already quite familiar with the windows world.

a silly unix program

unix has a number of interesting utilities. One that is handy is the cal command. Here's how it looks from the command line:
ds9{aharris}16: cal
   July 1999
 S  M Tu  W Th  F  S
             1  2  3
 4  5  6  7  8  9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
As you can see, when I type 'cal', I get a simple text version of this month's calendar. It would be nice to be able to deal with this calendar through perl, so we would not have to deal with getting the date formats ourselves. Here's a very simple example, that will work from the command line:
#!/usr/local/bin/perl
#cal.pl
#calls the unix cal command from within perl

#notice the accent characters.  NOT single quotes!!!!
print `cal`;
Of course, there is nothing at all interesting about this program from the user's point of view. If they could type 'cal.pl' they could also type 'cal' and run the program themselves. The main point of interest is the call to 'cal' inside accent characters. When perl encounters a string inside accent characters, it understands it to mean a command. The command will be executed, and the results will be returned back as a string, which we can print or do anything else we wish with. Of course, these features are already available to anyone using the command line. Such programs become more interesting when we re-write them as cgi programs, because the user would not normally be able to access programs residing on the server:
#!/usr/local/bin/perl
#cal.pl
#calls the unix cal command from within perl
#modified for CGI access

use CGI ':standard';

print header();

#notice the accent characters.  NOT single quotes!!!!
print start_html("this month's calendar");
print h1("this month's calendar");
print pre(`cal`);
print end_html();
This works, but it is still a bit ugly. We had to be sure the output was pre-formatted with the <pre> tag in order to get it to look right. The following code uses lots of cgi.pm tricks, array handling, and some pattern matches to capture the results of the command call into a variable, and to translate it into an HTML table.
#!/usr/local/bin/perl
#cal.pl
#calls the unix cal command from within perl
#modified for CGI access, more flexible output

use CGI ':standard', ':html';

print header();

#notice the accent characters.  NOT single quotes!!!!
print start_html("calendar");
print h1("What's wrong with this month?");
print h2("this month is accurate.  Why?");

#assign the calander to a variable
$month = `cal 9 1752`;

# turn the month into an array of weeks
@weeks = split (/\n/, $month);

#pull out the first line, it's the month and year
$title = shift @weeks;   #shift is like pop, but it takes from the front!
print h3($title);

#create a table
print " \n";

#make a row for each week
foreach $week (@weeks){
    #increment a line number counter
    $lineNum++;

    #change all spaces into : delimiters
    $week =~ s/\s+/:/g;

    #replace all leading : with nothing
    $week =~s /^://;

    #Turn the week into an array of days
    @days = split (/:/, $week);

    #check the number of days in this week
    $numDays = @days;

    # figure out how many holes we have in the week
    $padNumber = 7-$numDays;

    #for each of these spots...
    for ($i = 0; $i< $padNumber; $i++){
     if ($lineNum < 3){
        #if we're towards the beginning of the month, put spaces at beginning
        unshift @days, "<br> "; #unshift pushes to the FRONT of the array
      } else {
          push @days, "<br> ";  #add a space to the END of the array
      } #end 'beginning of month' if
    } # end for loop

  #print a table row containing the days
  if ($lineNum == 1){
    print Tr(th([@days]));
  } else {
    print Tr(td([@days]));
  } # end if 
} # end foreach
print "
\n"; print end_html();
There are a number of other very interesting programs that we can use to increase the power of our programs. Among the most interesting are command-line database applications. We can use these to send some kind of command to a database and return its value as a string or array which we can then manipulate in perl.

The dangerous part of this technique

While this technique of using programs that reside on the server is exceptionally powerful, it is extremely important that you do not allow the user to send commands to the server. NEVER write a program that blindly echos a command from a form to an accented string for execution. If you do, malicious users could use the program to harm your server, and it will all be done under your name!!

Sending email

One special application that is exceptionally useful is sending email. Most unix machines have a sendmail program which is designed specifically for command-line driven email access. You can use this to send emails from your cgi programs. Here's how you can send an email from the unix command line:
ds9{aharris}7: /usr/lib/sendmail aharris@cs.iupui.edu
from: me
subject: nothing really

Hi, me!!
How am I?

.

Sendmail is a program in unix, but since I do not need to use it frequently, I must give the entire path to this command. (I found the path with the 'which sendmail' command). I also had to specify who the email was to be sent to. Nothing appeared to happen, except I was able to type things. I did so, making sure that I included a from: line and a subject: line. Finally, I ended with a period on it's own, which sent me back to the command line.

To learn more about sendmail or any common unix command, consult the man pages ('man sendmail' at the command line) or look in a good unix reference.

Making a program write to sendmail:

Let's start with a very simple web page


<html>
<head>
<title>send mail tester</title>
</head>

<body>
<form method = "post"
      action = "http://klingon.cs.iupui.edu/cgi-bin/cgiwrap/aharris/webprog/mail.cgi">

<input type = "hidden"
       name = "recipient"
       value = "aharris@klingon.cs.iupui.edu">
From: 
<input type = "text"
       name = "from"
       value = "">
<br>
Subject: 
<input type = "text"
       name = "subject"
       value = "">
<br>

Message:
<textarea name = "message"
          rows = 10
          cols = 40>
This is defaut text.  It 
doesn't really matter what is

here.


</textarea>
<br>

<input type = "submit">
<input type = "reset">

</form>
</body>
</html>

This will gather all the necessary data, and send it on to mail.cgi, which looks like this:


#!/usr/local/bin/perl
#Change this line to point to your version of perl

use CGI ':standard';

#simple mailing program
# by Andy Harris, 7/99

#expects a form with a hidden field called recipient
#and a field called subject which can be hidden or text
#this field should have the name of the person who is being mailed to.
#one more field called message should have the main body of the message

#set up constant for mail program.  Change this to point to your version
#of sendmail
$mailprog = '/usr/lib/sendmail';

#get recipient from form
$recipient = param('recipient');

#get subject from form

$subject = param('subject');

#get 'from' address from form
$from = param('from');

#get the message
$message = param('message');

#send the mail
open (MAIL, "|$mailprog $recipient") || die ("couldn't open $mailprog! \n");
print MAIL "From: $from \n";
print MAIL "Subject: $subject \n\n";
print MAIL "Message: \n";
print MAIL $message . "\n \n";
print MAIL "(sent by mail.cgi, Andy Harris, 1999)";
print MAIL "\n \n";
close (MAIL);

#send a response to the user
print header();
print "<html><head><title>Email request Succcessful</title></head>";
print "<body><center><h1>Thanks for your input!</h1></center>";
print "recipient: $recipient <br> \n";
print "from: $from <br> \n";
print "subject: $subject <br> \n";
print "message: <br>  \n $message <br> \n";
print "<hr></body></html>";

How it works.

This program is relatively straightforward, except in the way it handles the mail. Let's start by looking at the familiar parts: We call up cgi.pm, read in a number of variables, and print an appropriate response out to standard output. In addition, we open up a special version of sendmail as a FILE, and print things to that file. This seems like a dirty trick, but it is very common in the unix world.

As you recall, <STDIN> and <STDOUT> are a lot like files to perl. This means that we can redirect input and output to other places than standard in and out. If we want, we can have programs print out to files. Look at this example:

ds9{aharris}1: ls -l > mylist
ds9{aharris}2: cat mylist
total 36
-rw-r--r--   1 aharris  staff         89 Jul 12 14:45 cal.pl
-rw-r--r--   1 aharris  staff          0 Jul 12 17:12 mylist
-rw-r--r--   1 aharris  staff       9823 Jul 12 17:12 otherStuff.html
-rw-r--r--   1 aharris  staff       6771 Jul 12 16:48 otherStuff.html~
When we did the ls command, we used the > character to redirect its output to a file called mylist. We did not see the data at all!!. The cat command is used to list out the contents of a file. This proves that the data was actually saved here. Unix allows all kinds of redirection. We are only interested in one more, called piping. You might by now have a number of files in your directories (especially public_html and cgi-bin). It could be too many files to see on one screen when you use the ls -l command. If you remember dos, there was a variant of the dir command that allowed you to pause at each page. Unix users do it like this:
 ls -l | more
(in fact, DOS users can and do the same thing sometimes, but there's not many real DOS users left anymore. sigh...) What this means is 'run the ls -l command, but pass it through the more command on the way. More is a special command which takes a number of lines and prints out only what will fit on the screen, pausing until the user presses a key. The | character is called the pipe, and this feature is called 'piping'.

All this piping and redirection is somewhat obscure, but it gives us incredible power, because we can redirect our output to any program that can accept a pipe. You might have guessed that sendmail is exactly such a program. If you look carefully at the open statement, you can see that we used a PIPE character, followed by the filename and parameter (which both happen to be variables). This tells us to open the file as a pipe, rather than directly for input or output. Now, it will act as a file open for output, but rather than saving the data in a file, it will send the information to the program. In essence, opening the file in this way replicates exactly the way we called sendmail from the command line.

Modifying existing programs

Almost all scripts need to be modified in some way before they will run on our system Most programs you get from the net will have good documentation on which variables to change and how to change them. Copy the source code to your cgi-bin subdirectory Give your program executable permission (711) Often there will be a related html file. Copy it to the appropriate subdirectory as well (Probably public_html) Change the first line so it points to klingon's copy of perl. This line MUST read the following: #!/usr/local/bin/perl Look through the perl source code in your editor. Look for comments that explain variables you need to change If in doubt, leave it alone. Change only what you HAVE to at first. The html document that refers to this script will require some changes as well. Make sure that wherever it refers to the script (most likely in the action attribute of the form) it has the correct URL for the script. Use the sample provided above, replacing aharris with your userid and foo.pl with the name of your script. Test, test, test!

Laboratory Assignment

In unix, the 'finger' command is used to return information about your current status. If I type 'finger aharris' on the command line, this will be the results:
Login name: aharris   			In real life: Andy Harris
Directory: /home/aharris            	Shell: /bin/csh
On since Aug 16 14:21:42 on pts/32 from 134.68.140.120
New mail received Wed Aug 18 14:23:25 1999;
  unread since Tue Aug 17 16:03:23 1999
No Plan.
Write a program that will run the finger program and extract from the results whether you are online and when you last checked email. This data should be returned as a nicely formatted web page. As always, please ensure that source code is available.
© Andy Harris
Indiana University / Purdue University, Indianapolis
email: aharris@.cs.iupui.edu
homepage: www.cs.iupui.edu/~aharris