25. Too Many Source Files!
Part of
CS:2820 Object Oriented Software Development Notes, Fall 2015
|
Over the past lectures, we've taken a single source file and broken it into multiple source files, then used Javadoc to create documentation. The result has one key downside, the huge number of files that choke the directory for the project. Here's what you get if you do an ls command on the directory for our road network simulation:
[dwjones@fastx08 ~/AAA]$ ls allclasses-frame.html RoadNetwork.java allclasses-noframe.html RoadNetwork.new classes ScanSupport.class constant-values.html ScanSupport.html deprecated-list.html ScanSupport.java Errors.class 'ScanSupport$Message.class' Errors.html ScanSupport.Message.html Errors.java 'ScanSupport$NotFound.class' example ScanSupport.NotFound.html exampleAB script.js help-doc.html serialized-form.html index-all.html 'Simulator$1.class' index.html 'Simulator$Action.class' Intersection.class Simulator.Action.html 'Intersection$ConstructorFailure.class' Simulator.class Intersection.ConstructorFailure.html 'Simulator$Event.class' Intersection.html Simulator.Event.html Intersection.java Simulator.html 'NoStop$1.class' Simulator.java 'NoStop$2.class' Sink.class NoStop.class 'Source$1.class' overview-tree.html 'Source$2.class' package-frame.html 'Source$3.class' package-list Source.class package-summary.html 'StopLight$1.class' package-tree.html 'StopLight$2.class' PRNG.class 'StopLight$3.class' PRNG.html 'StopLight$4.class' PRNG.java 'StopLight$5.class' 'Road$1.class' StopLight.class Road.class stylesheet.css 'Road$ConstructorFailure.class' testfiles Road.ConstructorFailure.html testscript Road.html testscripterrs Road.java testscriptout RoadNetwork.class testscripttemp RoadNetwork.html [dwjones@fastx08 ~/AAA]$
If you use a GUI to look at the directory (as a "folder"), the clutter is no better. We can break this hodgepodge of files down a bit by using selective ls commands. Here is the list of just the java source files:
[dwjones@fastx08 ~/AAA]$ ls *.java Errors.java PRNG.java RoadNetwork.java Simulator.java Intersection.java Road.java ScanSupport.java [dwjones@fastx08 ~/AAA]$
Here is the list of just the files created as output from the Java compiler:
[dwjones@fastx08 ~/AAA]$ ls *.class Errors.class 'Simulator$Action.class' Intersection.class Simulator.class 'Intersection$ConstructorFailure.class' 'Simulator$Event.class' 'NoStop$1.class' Sink.class 'NoStop$2.class' 'Source$1.class' NoStop.class 'Source$2.class' PRNG.class 'Source$3.class' 'Road$1.class' Source.class Road.class 'StopLight$1.class' 'Road$ConstructorFailure.class' 'StopLight$2.class' RoadNetwork.class 'StopLight$3.class' ScanSupport.class 'StopLight$4.class' 'ScanSupport$Message.class' 'StopLight$5.class' 'ScanSupport$NotFound.class' StopLight.class 'Simulator$1.class' [dwjones@fastx08 ~/AAA]$
Here is the list of just the HTML files created by Javadoc:
[dwjones@fastx08 ~/AAA]$ ls *.html allclasses-frame.html package-tree.html allclasses-noframe.html PRNG.html constant-values.html Road.ConstructorFailure.html deprecated-list.html Road.html Errors.html RoadNetwork.html help-doc.html ScanSupport.html index-all.html ScanSupport.Message.html index.html ScanSupport.NotFound.html Intersection.ConstructorFailure.html serialized-form.html Intersection.html Simulator.Action.html overview-tree.html Simulator.Event.html package-frame.html Simulator.html package-summary.html [dwjones@fastx08 ~/AAA]$
Nowhere in this mess is there a file explaining what all the files are. This is a bit of a problem.
Any time a large project reaches the point where there are too many files and no roadmap, we need to do something. In the UNIX world, the first solution to emerge was the README file (always given an upper-case name to make it stand out).
Simply add a file to the directory named README that explains what the directory contains and how to use it.
Unfortunately, there is no standard for the structure or contents of a README file, but obvious things to include are:
Some software distributions split these apart, so you have a file called COPYRIGHT and a file called AUTHORSHIP. Sometimes, there's a file called READMEFIRST that explains all the component "read me" files.
Here is a README file for the road network example:
A road network simulator By Douglas W. Jones This is freeware, you get what you pay for, so don't expect much SOURCE FILES IN THIS DIRECTORY Errors.java -- general purpose error reporting package PRNG.java -- general purpose pseudo-random-number generator ScanSupport.java -- support for Java's Scanner class Simulator.java -- general purpose discrete-event simulation framework Intersection.java -- intersections in the road network Road.java -- roads in the road network RoadNetwork.java -- main program classes -- the @classes file for the javac command TEST FILES IN THIS DIRECTORY example -- a small road network exampleAB -- a smaller one testscript -- a abandoned attempt at building a test script testfiles: -- a subdirectory used by the test script empty -- the empty file, used in the script test1-1errors -- the expecte errors from the first test The rest of the files in this directory were created automatically by the following shell commands: javac @classes javadoc @classes
The problem of building large applications is complex enough that people began to build tools to automate this in the 1970s. Perhaps the most general of these tools is the Make utility that grew out of the C and C++ communities. The thing that makes Make special is that it is not tied to C and C++, but can be applied to any programming language or mix of languages.
Traditionally, the makefile for a project should be called Makefile, assuming that there is only one project in the current directory. Makefiles allow comments with a # (pound sign) prefix, so all of the usual rules for commenting apply.
Here is a useful makefile for our project:
# Makefile
# tools for maintaining the road network simulator
# example uses
# make clean -- deletes automatically generated files from the directory
clean:
rm -f *.class *.html package-list script.js stylesheet.css
The basic structure of a make command is illustrated by the above. Each make command begins with the name of a target. To make the target clean, the make command will execute the shell command indented on the following line or lines. Classical versions of make require that all indenting in the makefile be done with tabs, spaces do not work.
By convention, most large projects have a make clean mechanism to delete everything that is automatically generated so that you can package up the contents of the directory for export, and so that you can clean out all the secondary files before you make a backup, and so that you can clean up the directory before you list it at the start of a day's work doing development.
Our project is intended to be used with Javadoc, so we can use the makefile to document how this is done as well:
# Makefile # input to make command to make things related to road network simulator # author Douglas W. Jones # example uses # make clean -- deletes automatically generated files from the directory # make html -- makes all the HTML documentation using javadoc html: classes javadoc @classes clean: rm -f *.class *.html package-list script.js stylesheet.css
We've added a new make target to our example uses, so if you type make html, it will run the command javadoc @classes. The added reference to classes after the colon on the target line tells make that, in order to make the target html, there must be a file called classes.
We can also tell make that it needs all the source files to make the html target by listing them on the same line. Such file lists get long, but we can use make variables to clean this up and add additional documentation:
# Makefile # input to make command to make things related to road network simulator # author Douglas W. Jones # example uses # make clean -- deletes automatically generated files from the directory # make html -- makes all the HTML documentation using javadoc # all the source files, by category supportFiles = Errors.java PRNG.java ScanSupport.java Simulator.java modelFiles = Intersection.java Road.java mainFile = RoadNetwork.java JavaSourceFiles = $(supportFiles) $(modelFiles) $(mainFile) # make targets html: classes $(JavaSourceFiles) javadoc @classes clean: rm -f *.class *.html package-list script.js stylesheet.css
Note that the variable names we've used for each group of file names documents what the names in that group do. Note also that make uses a really miserable notation for references to named variables, a $ (dollar sign) prefix on a parenthesized variable name. This makes variable names stand out where they are used, but it is hardly convenient or intuitive.
All make variables are character strings, typically just lists of file names separated by blanks. Variables must be defined before use.
Now, we can add a make target for the whole application:
RoadNetwork.class: classes $(JavaSourceFiles) javac @classes
Since this is the primary make target for our application, we should list it first. By default, if you just type make, it will make the first target, so by listing this one first, typing make is equivalent to typing make RoadNetwork.class.
Whicever one you use, make is smart enough to check if any of the files on which the target depends have been modified after the date of last modification of the target. If they have been modified, it will run the shell command. If not, it will just output a message saying that the target is already up to date.
At this point, we aren't really using the inter-file dependencies that exist in our collection of files. To start using these, note that our makefile already provides better documentation of the source files than the classes file because we've grouped the files by category and used variable names that identify the category. As a result, we can use this as our primary list of file names and let make automatically generate the classes file:
classes: echo $(JavaSourceFiles) > classes
This says that to make the file classes, if this file does not already exist, generate it using the a shell command that creates the file from the list of java source files inside this makefile.
For each make target, make includes a check to see if the target is up to date, so if the date of last modification of the file classes is after the date of modification of all of the files on which it depends, it will not re-make it, while if any of those files have been changed since classes was last modified, make will re-make it.
Since the contents of classes was created from the makefile, we should really include Makefile in the list of dependencies. This way, if someone edits the makefile, then classes will be automatically regenerated the next time it is needed.
We can now include classes in the list of files deleted by make clean, giving us this makefile:
JavaSourceFiles = $(supportFiles) $(modelFiles) $(mainFile) # make targets classes: Makefile echo $(JavaSourceFiles) > classes html: classes $(JavaSourceFiles) javadoc @classes clean: rm -f *.class *.html package-list script.js stylesheet.css
We are still missing something: Make can also do the primary job of compiling our source files. When we run the program, we run it as RoadNetwork, which is taken to be a reference to the file RoadNetwork.class, so that is the make target for the application. Make treats the first target as the default, so if you put RoadNetwork.class as the first make target in your list of targets, just typing make becomes equivalent to typing make RoadNetwork.
It is useful to separate the make targets into categories: The primary target, subsidiary make targets needed in order to make the primary target, and secondary targets you may never need to use. That is done in the following version of the makefile:
# Makefile # input to make command to make things related to road network simulator # author Douglas W. Jones # primary use # make -- make RoadNetwork.class, the Java executable for the simulation # make RoadNetwork.class # secondary uses # make clean -- deletes automatically generated files from the directory # make html -- makes all the HTML documentation using javadoc # all the source files, by category supportFiles = Errors.java PRNG.java ScanSupport.java Simulator.java modelFiles = Intersection.java Road.java mainFile = RoadNetwork.java JavaSourceFiles = $(supportFiles) $(modelFiles) $(mainFile) # primary make target RoadNetwork.class: classes $(JavaSourceFiles) javac @classes # subsidiary make targets classes: Makefile echo $(JavaSourceFiles) > classes # secondary make targets html: classes $(JavaSourceFiles) javadoc @classes clean: rm -f *.class *.html classes package-list script.js stylesheet.css
The first time you type make, it will begin by creating the classes file and then it will run javac @classes. If you do nothing but type make again, it will tell you that RoadNetwork.java is up to date and not remake it. If you edit any of the source files and then type make, it will notice that you changed something on which RoadNetwork.java depends and it will recompile everything.
The behavior is not as nice for regenerating the HTML files. The reason is, the make command for target html doesn't create a file with that name, so every time you ask it to make html, it will repeat the job. We can fix this as follows:
index.html: classes $(JavaSourceFiles) javadoc @classes html:index.html
This explicitly names the primary output of Javadoc so that make can compare the date of last modification of index.html with the dates on all the files it depends on. If you just type make html, there are no shell commands to execute but because it depends on index.html, you asked make to make sure that is up to date.
The top level documentation at the head of the makefile doesn't need to list secondary make targets, but the structure of the file dependencies between the secondary targets is documented by the makefile.
We can easily add test scripts to the makefile as secondary make targets:
tests: RoadNetwork.java exampleAB example java RoadNetwork exampleAB java RoadNetwork example
Now, we can run all of our tests by typing make tests. If we'd followed through on our original development plan by creating and maintaining test scripts, this could launch those test scripts instead of directly running the tests.