[sclug] Software development strategies for the lazy

[pieter claassen]

Hello All,

I have just been through another cycle of software development in
isolation and thought that it would be great to have a conference to
talk with other software developers about how they develop software and
what they think are *really* good and *really* bad ways of developing
software.

Alas, as groundwork to such a possible meeting, I thought I would write
a quick email on what I have learned with this latest project and would
appreciate any comments from other developers (who I am sure have
engaged in more complex problems then this!)

The requirements for the app are:
     1. It must be secure.
     2. Accessible remotely/locally.
     3. Accessible via a GUI and command line.
     4. Glue together a large number of existing programs written in a
        variety of languages. (use existing apps)
     5. Augment these programs with custom functionality (do new
        things).
     6. Be flexible because the underlying technology is OSS and will
        change.
     7. Be well documented.
     8. Must easily be scaled to a larger number of developers.

In the end, the strategy I decided on was:
     1. Browser based solution.
     2. CGI written in Perl.
     3. CGI only uses command line utility wrappers written in bash/perl
        or whatever needed and never talks to the system directly.

No rocket science so far, but this is what I learned from the project:
     1. The bash/perl command line utilities was a good idea because the
        main objective is to wrap existing functionality with default
        parameters relevant to the program. This means that the learning
        curve for users is quite low (flexibility trade-off as usual
        though). e.g. to configure and enable a bridge would require at
        least 5 commands with a large number of command line options.
        Because a bridge can only be used in a certain way in the
        software, this is all replaced with a single program called
        bridge_control that takes either "on" or "off" and that does the
        rest in the background.
     2. Wrapping the CGI environment around the command line utilities
        by calling them through system() and `` calls works well because
        it allows programming by contract. As long as the way that you
        interface with the wrappers doesn't change, then you have
        effectively separated presentation from logic. You can then
        change the CGI environment which is now quite simple without
        bothering to modify the way the applications work and vica
        versa.
     3. You can also disable all privileged functionality by just moving
        the wrapper scripts (disabling them). This allows for quick
        harmless demos with anonymous access without breaking the CGI
        environment.
     4. You can easily write a Perl/TK local user front-end that
        utilises the same wrapper scripts without breaking your version
        control. This allows you to develop both browser based software
        as well as an X windows application while still utilising the
        bulk of your code and maintaining a command line interface.
     5. However, it doesn't make sense to put everything through the
        command line because some things make no sense. e.g. the editing
        of a configuration file that was designed to be edited by hand
        in the first place. If you are in command line mode, then you vi
        the file. Also, things like trend analysis that produces graphs
        that can only be viewed in the gui anyhow. This means that even
        though the bulk of your code runs as a non-privileges user that
        calls your code with sudo, you still need to write some logic
        directly into the CGI environment. I found the way around this
        was to make sure to try and stick to OO perl and pull that logic
        out in separate modules.
     6. As far as security for the application is concerned, a big issue
        was the fact that the system will now be running a webserver
        locally and you need to make sure that it stays up to date. So,
        as long as unauthorised users cannot log onto the system, then
        you should be ok. Once you are logged on to the system, you
        effectively have root privileges through the browser. Not sure
        if this is really an issue.
     7. Perl as a development environment does provide the flexibility
        required to stay on top of the changes in the source
        applications. It would be difficult for me to move to Java or
        even C because I cannot see the benefit of the conger and more
        complex development process.
     8. Documentation is being done in POD for the software and a wiki
        for user docs which means it is mainly text that gets marked up
        automatically. The wiki is installed locally with the
        application and there is a CGI script to automatically update
        the documentation. I am using twiki which is not bad at all.
        However, I am not totally sure that this proprietary markup is
        the right way to go. The main issue is that if you ever need to
        access the content directly for whatever reason, then you can
        read it in any editor, but you will need to write a parser to
        make it look a little better. However, it is possible write HTML
        directly in the wiki and I might consider that. The problem with
        documentation is that I need something that makes it easy to
        cross reference, produces HTML and is portable. A simple HTML
        editor might work better. Documentation is still a little
        headache.

Has anybody ever developed serious software with Perl/Tk and what were
your experiences?

Any comments appreciated.

Pieter