ISRL CGI Development FAQ

• On which systems can I run CGI's?
• On which systems should I delelop CGI's?
• Where should files be placed so that they are accessible by the correct web server?
• I created a cgi and put it under the proper subdirectory, but all I get is the text of the script ... the script does not execute.
• How can I be sure my script is running properly? (that is, where are the log files located)?
• When I try to run my script, I get a web page that says "Internal Server Error". What's wrong?
• I want to use the script to upload files. Are there any issues I should be aware of?
• The error log says the script doesn't have permission to do something. What do I do?
• Can I password protect access to the cgi script?
• Can I use SSL with my script to encrypt password/data transfers
• What languages can I use to create CGI's?
• What about Java and PHP?

On which systems can I run CGI's?

While all the ISRL systems that have running web servers allow CGI's to be run, CGI development should only occur on soldev.isrl.uiuc.edu or www3.isrl.uiuc.edu. Soldev is designed for Solaris development and www3 is designed for linux development. If you are starting a new project, www3 is recommended. Only stable CGI's that require high availability should run on www.isrl.uiuc.edu. ALL CGI's and other dynamic HTML delivery mechanisms are more resource intensive than static HTML pages, particularly those that are under development and can end up out of control as a runaway process. (See also http://www.isrl.uiuc.edu/systems/Documentation/Which_Web_Server.html.)

On which systems should I delelop CGI's?

On www3.isrl.uiuc.edu or perhaps soldev.isrl.uiuc.edu ... see answer above.

Where should files be placed so that they are accessible by the correct web server?

The web files for www.isrl.uiuc.edu should be placed in the public_html within an accounts web directory. This should be something like /content/web/users/username/public_html. The web files for soldev.isrl.uiuc.edu should be placed in /content/web2/users/username/public_html. Files for www3.isrl.uiuc.edu should be placed in /content/web3/users/username/public_html. For convenience, symlinks are created (like shortcuts on windows) that allow these directories to appear in home directories. For instance, files placed in

  /home/mwolske/public_html  (which is a link to /content/web/users/mwolske/public_html)

will be available via the URL:

  http://www.isrl.uiuc.edu/~mwolske

While files placed in

  /home/mwolske/public_html2  (which is a link to /content/web2/users/mwolske/public_html)

will be available via the URL:

  http://soldev.isrl.uiuc.edu/~mwolske

And files placed in

  /home/mwolske/public_html3  (which is a link to /content/web3/users/mwolske/public_html)

will be available via the URL:

  http://www3.isrl.uiuc.edu/~mwolske

Some accounts are also accessible via a virtual domain. For instance,

  /home/inquiry/public_html

would be accessible at either of the following URL's

  http://www.isrl.uiuc.edu/~inquiry
  http://www.inquiry.uiuc.edu/

For clarity, best practice suggests you should put images in an images subdirectory and cgi's in a cgi-bin subdirectory, although nothing on our web servers enforces such a practice. For instance, a CGI called hello.cgi placed in the subdirectory

  /home/mwolske/public_html3/cgi-bin/hello.cgi

would be accessed via the URL

  http://www3.isrl.uiuc.edu/~mwolske/cgi-bin/hello.cgi

I created a cgi and put it under the proper subdirectory, but all I get is the text of the script ... the script does not execute.

Any CGI must end in .cgi for it to be interpreted by the Apache web server as a CGI. Furthermore, the CGI should have be executable by everyone. To change the permissions, use the command:

  chmod 0775 scriptname

How can I be sure my script is running properly (that is, where are the log files located)?

Since users cannot login to www.isrl.uiuc.edu, you can't access the log files directly. However, there is a simple script that you can use to at least view the last 20 lines of each of the log files. Since this data can be used to compromise the system, it's only accessible to authenticated ISRL users. It can be found at https://www-s.isrl.uiuc.edu/systems/isrl-only/report.cgi

On www3.isrl.uiuc.edu and soldev.isrl.uiuc.edu, you can access the logs in /var/log/apache directly, so you can view the entire logs. Whenever making modifications to CGI's on these systems, please take the following steps to monitor the log file while testing:

1. Open a second SSH window to www3 or soldev and login.
2. To start a continuous monitoring of the error log, at the command prompt, type:

            tail -f /var/log/apache/error_log
     

3. To exit the continuous tail when you finish testing your changes, type:

            Control-c 
     

You can repeat the above steps, substituting access_log for error_log to monitor the successful accesses to a web page, along with information about the type of client used, the referer page, and the requesting IP address. Thus, by having 3 SSH windows open, you can be changing code and monitoring both the successful accesses and errors in the script.

On www.isrl.uiuc.edu, the general logs are not directly accessible. However, we can have the logs for individual virtual domains (e.g., www.dkrc.org which is aliased to the www.isrl.uiuc.edu web server) can be written to a logs subdirectory for that account.

When I try to run my script, I get a web page that says "Internal Server Error". What's wrong?

The first thing to do is to check the error log for the web server. On soldev.isrl.uiuc.edu and www3.isrl.uiuc.edu, you can get a running log of the error log by logging in to soldev and typing the command:

  tail -f /var/log/apache/error_log

To stop the output of the log, type ^C (that is, hold down the control key and hit the "c" key). Scripts can fail for many reasons. By looking at the error logs, you should get a good indication of what is making it fail. If it appears to be related to the web server configuration, please email isrl-support@isrl.uiuc.edu . Include in the message the lines from the error_log related to execution of the CGI.

I want to use the script to upload files. Are there any issues I should be aware of?

Net users can and do use these free repository spaces to share files when they discover such places. And because of search engines, it can often be easy to find such free repositories. To avoid this sort of use, the following practices should be followed:

• Place the uploaded files into a subdirectory outside of the public_html tree. For instance, create a subdirectory in the accounts home directory called upload files, change it's permission to be world readable, and write the files there. Commands to do this would be (do not type the # sign or comments which follow the # sign)

  
      cd                    # changes to your home directory
      mkdir dirname         # creates a subdirectory called dirname in your home directory
      chmod 777 dirname     # gives the world read/write permissions on this subdirectory
  

• If the files need to be accessed by others, use a second CGI to read and deliver those files. The script should be designed to only display expected files. That is, make sure a hacker cannot upload Warez (that is, illegally copied software) and advertise the existence of those warez by whatever name to their friend.
• Uploaded files should never have their execute bit turned on. Hackers can and do use such holes to upload and execute malware, especially if files are writeen to the public_html directory.

The error log says the script doesn't have permission to do something. What do I do?

ALL CGI's run as user nobody. If the script is trying to write a file, the file must be world writeable (see above). If the script is trying to execute a process, that process probably can't be executed by the user nobody. If this is the case, there's not much you can do.

You CANNOT change the ownership of the CGI and have it run as a user other than nobody. You CANNOT set the UID sticky bit and have the script run as another user. The CGI will always run as user nobody. If the script is asked to run a process that cannot be executed by user nobody, it will not run. For instance, to do a file write, the subdirectory to which the file is being written must be world writeable (and therefore world deletable). This is a critical security setting that cannot be changed.

Can I password protect access to the cgi script?

Yes. The easiest way to do this is using a .htaccess file in the cgi-bin subdirectory. (FYI, this mechanism can also be used to make all files in the cgi-bin subdirectory execute as a CGI, not only those ending in .cgi.) There are a number of good references on setting up and configuring a .htaccess file. For instance:

• The HTAccess Authentication Tutorial
• Using .htaccess Files with Apache

Can I use SSL with my script to encrypt password/data transfers

Yes. Place files in /content/web/users/<username>/public_htmls. These files are then available from

https://www.isrl.uiuc.edu/~<username>

Note the https, not http.

What languages can I use to create CGI's?

ISRL systems support Perl, shell (Bourne, BASH, and C-shell), and 'C' CGI's. Unless the CGI is a compiled binary written in 'C', you'll need to have the first line of the script specify which language to use. On ISRL, these would be:

  PERL   - /usr/bin/perl (may be different on Solaris)
  BOURNE - /bin/sh
  BASH   - /bin/bash
  CSH    - /bin/csh

What about Java and PHP?

While the latest versions of these are certainly supported on the ISRL systems, they are technically not CGI's and therefore many of the issues related to these two languages are different from those for CGI-based languages.