[dundee] Concurrent Programming on Linux

[Gary Short]

Hello Rick,

Thanks for this feedback, it's very helpful. I have a (I think) great idea
for a Linux based network appliance and so I'm concerned more with taking
advantage of the multi-cores on the appliance than I am with scaling up or
out. Looks like I'll be taking a look at Clojure.

Thanks again,
Gary

PS - Yes I know I top posted, deal with it. :-P LOL!

-----Original Message-----
From: dundee-bounces at lists.lug.org.uk
[mailto:dundee-bounces at lists.lug.org.uk] On Behalf Of Rick Moynihan
Sent: 18 November 2009 10:04
To: Tayside Linux User Group
Subject: Re: [dundee] Concurrent Programming on Linux

2009/11/17 Gary Short <gary at garyshort.org>:
> All,
>
> Which is the best programming language to use for concurrent programming
on
> Linux? By best I also mean one that has a rich library for handling JSON
and
> XML input too - I don't want to go back to the RegEx stone age. J

Hey Gary,

Good question, and as is always the case it depends specifically upon
the details of the problem, and what you think is important.  What do
you mean by best?  How important are:

- Speed of sequential execution
- Scaling up across cores
- Scaling out across machines
- Expresivity, familiarity, interop, programming paradigm etc

Regardless here's some suggestions:

- Clojure

  + JVM based (best library support of any platform - seamless Java interop)
  + Functional language, immutable data
  + All reference types have strong concurrency semantics
     + Atoms, CAS style atomic updates on single values
     + Refs, Built in Software Transactional Memory (STM) for
co-ordinated, synchronized change...
     + Agents, for uncoordinated, unsynchronised change
     + Vars, for thread-local bindings
     + Assumes local concurrency model (i.e. you can safely peek at
values without blocking or freezing the world)
       - Terracotta (distributed JVM) can be used for distribution
  + Fast (maybe 3-4x quicker than python) in the general sequential case
  + Lisp (great metaprogramming... macros, parentheses etc...)
  + Dynamically Typed
  + Lots of good XML/JSON parsers available
  + Multi-methods

http://delicious/InkyHarmonics/clojure+video

- Erlang

  + Very good at parallelism, multi-core and scaling out
  + Functional language, immutable data
  + Actor model
     + Message Passing
     + Concurrency model is ALWAYS distributed (even when its local) -
imposed complexity
  + Robust & fault tolerant
  + XML/JSON parsers
  + Clunky string processing
  + Dynamically Typed
  + Pattern Matching
  + Excellent error handling... crash, burn and restart philosophy...
  + OTP - framework for building HA systems

http://delicious/InkyHarmonics/erlang+video

Of the above Clojure is my favourite, and is more suited to general
purpose workloads than Erlang...  Rich Hickey, the creator of Clojure
has proven to be an excellent ambassador of the language, providing
lots of good tech talks on clojure (all recorded).

If I was writing mission critical High Availability stuff, Erlang
would be my first choice.  Other languages that you might consider
(but I know less about include):

- Objective Caml

  + Incredibly fast....
  + powerful ML static type system
  + Immutable Data

- Haskell

  + Purely functional, immutable data
  + Very Strongly typed
  + Fast
  + lazy evaluation
  + Very academic but has XML/JSON parsers and is used in the real world.
  + STM, lots of concurrency stuff...

- Scala

  + JVM Based (awesome library support via Java... pretty good java interop)
  + Actors
  + multi-paradigm
  + Strongly typed
  + Mostly just "a better Java/C#"
  + As it doesn't favour immutability it's IMHO less suited to
handling concurrency problems.

Alternatively if you don't actually care too much about language level
support for parallelism you can get it (at the expense of lots of
memory) by pushing it to the O/S.... i.e. by making use of Unix system
calls for IPC, e.g. fork, pipe, kill, select etc... you can start a
bunch of separate O/S processes... This has been done recently with
great success in Unicorn (an almost pure Ruby HTTP Server), and is an
approach that can be adopted in most languages.

There is also stackless python, which might be of interest as it
simulates LOTS of parallelism with coroutines/continuations etc...
However the Global Interpreter Lock (GIL) means that you'll never be
able to take advantage of multiple cores... meaning its main use is if
you spend a lot of time waiting on I/O.

All of these languages work excellently on Linux... they probably also
work well on Windows too.

No doubt see you soon mate,

R.

_______________________________________________
dundee GNU/Linux Users Group mailing list
dundee at lists.lug.org.uk  http://dundeelug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/dundee
Chat on IRC, #tlug on irc.lug.org.uk

!DSPAM:5,4b03c6c8200756733420660!