[sclug] Software development strategies for the lazy

[Paul Vanlint]

Hi Pieter,

This is a pretty big area to discuss, but I would enjoy taking part in
such a meeting. I would prefer somewhere outside of town as parking can
be a pain. Perhaps a nice pub south of the M4, as I live in Basingstoke?

The subject of software engineering is a pretty big one and Pieter has
only discussed a very small part here, which is very particular to his
requirements.

It probably helps for people to describe their background and experience
in this area, so we know where we are.

I have worked on commercial software releases for about 10 years now and
have been developer, QA, project manager, engineering manager and
architect at various times.

One of the key goals of any development technique should be to maximize
the efficiency of all involved. I think that the two biggest time
wasters I have come across were large bug fixing cycles and post-design
requirement changes which meant either refactoring code, or trying to
fit a square peg into a hole that keeps getting rounder and rounder.

Where possible write reusable code, minimize the bugs generated in the
first place and identify and fix bugs as early as possible.

There are a number of principles that can help here, but the key ones
that spring to mind are:
1) Modularise the code as much as possible.
2) Decouple the modules as much as possible.
3) Document interfaces well
4) Unit test the code as early and as often as possible. If a change
breaks something then the sooner that bug is noticed the easier it is to
fix.
5) Fail early. This means to use things like assertions so that you can
detect bugs as soon as they happen in the code and can give some hint as
to where the bug is, rather than letting a bug hide until later on in
the program at which point it would be very hard to find the actual bug
causing the effect.
6) Make the compiler/system do as much bug finding as possible. This
means turn on all warnings in compilers and adopt a style so you can
write the code in such a way as to avoid common pitfalls. For instance,
in C++ a common error made is to forget to use == in a comparison and
use = instead. If you write it like this
if ( 5 == var ) ...
Then using a single = will cause the compiler to spot the error.
7) Document the code well. This is a big subject in itself, but the
common problem is that either there are no comments or that the comments
become out of date and are not updated when the code changes. Wrong
documentation is generally worse than no documentation. Wherever
possible make the code self documenting, use function names and
variables that make sense. Use of an automation tool to generate
documentation is useful too.

Oh and the other thing to bare in mind is ALWAYS use some kind of
revision control on the software, so that you can find out what has
changed when bugs occur.

There is a book that I have read a couple of times that I would
seriously recommend to everyone. It is called "The Pragmatic Programmer"
and really provides a lot of useful ideas.

As far as particular languages or particular architecture designs, these
should be very much tailored to the particular requirements or the
skillset of the engineers involved.


I have developed software in a number of languages and here are some of
my experiences, in no particular order:

Assembly Language - Fun, but slow and prone to bugs that can be hard to
find. Last things I did in this were I/O routines for a network device
driver and some optimized FPU code for a scientific project
C - Very flexible and powerful, but with great power comes great
responsibility, so I still feel it is a language best used by a more
disciplined, experienced programmer for large commercial projects.
C++ - I like C++, very flexible and well thought out, but
troubleshooting can be harder than in C as there are more things done
'behind the scenes'. One problem is that junior programmers tend to
write predominantly C and call it C++.
Java - Similar to C++ in many ways, but I don't feel that I can get
quite the flexibility or performance I like.
Perl - Quite a good 'get things done' language but the syntax can make
it hard to follow for other engineers to maintain and similarly can make
it hard for junior programmers to pick up. So I would say not a great
language to use for a large project, but good for gluing things
together. Using the Perl modules provides a great amount of
functionality.
PHP - My current favourite for server side web development. Has many
powerful features built in. The weak typing in this language can lead to
tricky bugs to find for the junior programmer.
BASH - Good for small scripted things, but don't let it get too big.
Python - Still early days for me, but I quite like using the indentation
to identify blocks
BASIC - Most recently VB. I feel this is fine for prototyping, but, at
least in the past, suffered from some quirks which made troubleshooting
a bit of a pain.
FORTRAN - Used a long, long time ago, due to needing to use the NAG
library. Would not choose to use it today.

Hopefully this sparks some discussion.

Regards,

Paul.

On Tue, 2003-05-06 at 10:12, pieter claassen wrote:
> Hello All,
> 
> I have just been through another cycle of software development in
> isolation and thought that it would be great to have a conference to
> talk with other software developers about how they develop software and
> what they think are *really* good and *really* bad ways of developing
> software.
> 
> Alas, as groundwork to such a possible meeting, I thought I would write
> a quick email on what I have learned with this latest project and would
> appreciate any comments from other developers (who I am sure have
> engaged in more complex problems then this!)
> 
> The requirements for the app are:
>      1. It must be secure.
>      2. Accessible remotely/locally.
>      3. Accessible via a GUI and command line.
>      4. Glue together a large number of existing programs written in a
>         variety of languages. (use existing apps)
>      5. Augment these programs with custom functionality (do new
>         things).
>      6. Be flexible because the underlying technology is OSS and will
>         change.
>      7. Be well documented.
>      8. Must easily be scaled to a larger number of developers.
> 
> In the end, the strategy I decided on was:
>      1. Browser based solution.
>      2. CGI written in Perl.
>      3. CGI only uses command line utility wrappers written in bash/perl
>         or whatever needed and never talks to the system directly.
> 
> No rocket science so far, but this is what I learned from the project:
>      1. The bash/perl command line utilities was a good idea because the
>         main objective is to wrap existing functionality with default
>         parameters relevant to the program. This means that the learning
>         curve for users is quite low (flexibility trade-off as usual
>         though). e.g. to configure and enable a bridge would require at
>         least 5 commands with a large number of command line options.
>         Because a bridge can only be used in a certain way in the
>         software, this is all replaced with a single program called
>         bridge_control that takes either "on" or "off" and that does the
>         rest in the background.
>      2. Wrapping the CGI environment around the command line utilities
>         by calling them through system() and `` calls works well because
>         it allows programming by contract. As long as the way that you
>         interface with the wrappers doesn't change, then you have
>         effectively separated presentation from logic. You can then
>         change the CGI environment which is now quite simple without
>         bothering to modify the way the applications work and vica
>         versa.
>      3. You can also disable all privileged functionality by just moving
>         the wrapper scripts (disabling them). This allows for quick
>         harmless demos with anonymous access without breaking the CGI
>         environment.
>      4. You can easily write a Perl/TK local user front-end that
>         utilises the same wrapper scripts without breaking your version
>         control. This allows you to develop both browser based software
>         as well as an X windows application while still utilising the
>         bulk of your code and maintaining a command line interface.
>      5. However, it doesn't make sense to put everything through the
>         command line because some things make no sense. e.g. the editing
>         of a configuration file that was designed to be edited by hand
>         in the first place. If you are in command line mode, then you vi
>         the file. Also, things like trend analysis that produces graphs
>         that can only be viewed in the gui anyhow. This means that even
>         though the bulk of your code runs as a non-privileges user that
>         calls your code with sudo, you still need to write some logic
>         directly into the CGI environment. I found the way around this
>         was to make sure to try and stick to OO perl and pull that logic
>         out in separate modules.
>      6. As far as security for the application is concerned, a big issue
>         was the fact that the system will now be running a webserver
>         locally and you need to make sure that it stays up to date. So,
>         as long as unauthorised users cannot log onto the system, then
>         you should be ok. Once you are logged on to the system, you
>         effectively have root privileges through the browser. Not sure
>         if this is really an issue.
>      7. Perl as a development environment does provide the flexibility
>         required to stay on top of the changes in the source
>         applications. It would be difficult for me to move to Java or
>         even C because I cannot see the benefit of the conger and more
>         complex development process.
>      8. Documentation is being done in POD for the software and a wiki
>         for user docs which means it is mainly text that gets marked up
>         automatically. The wiki is installed locally with the
>         application and there is a CGI script to automatically update
>         the documentation. I am using twiki which is not bad at all.
>         However, I am not totally sure that this proprietary markup is
>         the right way to go. The main issue is that if you ever need to
>         access the content directly for whatever reason, then you can
>         read it in any editor, but you will need to write a parser to
>         make it look a little better. However, it is possible write HTML
>         directly in the wiki and I might consider that. The problem with
>         documentation is that I need something that makes it easy to
>         cross reference, produces HTML and is portable. A simple HTML
>         editor might work better. Documentation is still a little
>         headache.
> 
> Has anybody ever developed serious software with Perl/Tk and what were
> your experiences?
> 
> Any comments appreciated.
> 
> Pieter
> 
> 
> _______________________________________________
> sclug mailing list
> sclug at sclug.org.uk
> http://www.sclug.org.uk/mailman/listinfo/sclug
>