[Gllug] ulimits

[Nix]

On Sun, 2 Sep 2001, Alex Hudson said:
> On Sun, Sep 02, 2001 at 10:16:11AM +0100, Jon Masters wrote:
>> > You can't just stop a process. It doesn't work like that. Try
>> > implementing it on any half-complex program and watch your machine lock
>> > up solid.
>> 
>> That's the challenge.
> 
> And obviously this problem, which has evaded some of the best minds in
> hard realtime research will be no problem for the mighty jcm ;)

He's not looking for something which satisfies hard realtime
constraints; just an `oops, you're too CPU-hungry, SIGSTOP'. Threads are
fairly tricky and I wouldn't expect all variants of clone() to be
caught; but the common CLONE_VM LinuxThreads case can definitely be
caught and all sibling threads (and the parent thread manager) can be
SIGSTOPPed too. (Summing the CPU times of all the threads is, er,
trickier.)

It's not that hard; there's already code that checks for overmuch
CPU-time usage and sends SIGKILLs, and nobody complains about
that. (Admittedly, it hardly ever gets *used* either, because SIGKILL is
such a blunt axe...)

>> ...and what am I studying? Flying pink pandas? It was compsci last time
>> I checked. Further, I did happen to spend some time last year reading
>> through a couple of the more popular books which include an analysis of
>> scheduling.
> 
> Wow, you're an undergrad and you've read some books about it? Well, even
> though this is doctorate material you're obviously more than qualified.
> 
> Note the adjective - 'hardcore'. As in, something beyond the abilities of an
> undergrad. I wasn't infering it wasn't in your area, I was infering it was a
> problem beyond your abilities.

Curious, really; many of the people hacking on the Linux scheduler are
undergrads (the redoubtable Andrea Arcangeli for one). By this logic, it
must be beyond their abilities.

(As far as I can tell, very little is beyond Andrea's ability, the man
is amazing; he's a bit prone to hack before thinking the problem through
but then I'm just like that too so I can't complain ;) )

>                                Given you posed the original question means
> you have no grounding in this area whatsoever: the people currently working
> on solutions have been working on it for years, it's cutting edge, and it's
> not something you can just pick up. 

Er, understanding the Linux kernel scheduler and the reasons behind its
design requires just a good bit of time, a good memory, and a good head
for detail; it's not rocket science. Understanding the kernel's tangle
of locks and so forth is much harder, and I won't even *mention* the VFS
and the hell that is do_rename() and its locking constraints.

>> > I also don't see the point of cutting off users programs at arbitrary
>> > limits - what a waste of CPU cycles.
>> 
>> Yup indeed, especially when a user consumes 100% CPU for 500 squillion
>> hours because they forgot to account for condition X occuring.
> 
> And this is a problem how? I presume you know how nice works. Unless you're

It's a problem because you might want to stop the process and send the
user an email, say, if you knew that the process wasn't going to consume
more CPU than that. You can do something similar with setrlimit(), but
it's not the same...

> worried about the power drain of the processor executing real code as
> opposed to idling... 

This drain may not exist; e.g. the UltraSparc CPUs have no low-power
mode. (Does the Alpha?)

>                                                       So, setting up memory
> limits etc. make sense, and this is something the OS can manage effectively. 

... although, again, with the blunt axe of SIGKILL...

>> > If you are actually serious about this area of research you're on the wrong
>> > operating system.
>> 
>> Without meaning to be rude, who are you to tell us we are on the wrong
>> OS?
> 
> Linux is not a research OS.

Yes it is. The *official Linus kernels* aren't, sure, but there's
absolutely nothing stopping you from using your own copy of the source
tree for whatever you want, including OS research.

(In fact, its modularity may assist you there.)

>                             Find me a paper researching OS principles on
> Linux (not 'an implementation of x principle on Linux', but, 'new principle
> X with example implementation for Linux') and I'll show you twenty on Mach.
> Mach is a research OS.

Unfortunately, Mach is also an overfeatured crawling horror :( the Hurd
people have stated that basing the Hurd on Mach was a mistake and that
it has probably added several *years* to the Hurd's development time
(sorry, no URL; it was on linux-kernel, about a year ago).

>                         That's what it was built for. That's what it's good
> at. People writing Resource Kernels do it on Mach.

Actually they use other things these days; Mach is too
horrible. Exokernels are the thing :)

>                                                    Linux just isn't suited
> for that kind of work.

Wrong. It has a very simple schedule() function, and it is (fairly) easy
to follow the effects of any given change to it, and to benchmark the
effects of a change to it. It's a damned useful testbed for alternative
schedulers.

(But note that Linus has pointed out that for scheduling, the time
constant of the algorithm used *matters*, because the run queue is
normally so short. So an O(log n) algorithm could be worse than an
O(n^2) algorithm if the O(log n) one has a big time constant...)


Further, Mach's scheduler is in the microkernel. Mach's microkernel is,
er, not very nice to work on. Certainly much nastier than
kernel/sched.c. Scheduling and MM are about the only things in Mach you
must hit the microkernel for, but they *are* there... :(

> If you then manage to get a system that works, by all means port it to
> Linux. But it's much simpler to start on Mach. And it's so unlikely that

I disagree strongly :(

> you'll ever write something which works in the way you want, even on Mach,
> that the question of the port becomes moot :P.

All you need is persistance and (a lot of) care. It's not that hard.

-- 
`It's all about bossing computers around. Users have to say "please".
Programmers get to say "do what I want NOW or the hard disk gets it".'
                        -- Richard Heathfield on the nature of programming

-- 
Gllug mailing list  -  Gllug at linux.co.uk
http://list.ftech.net/mailman/listinfo/gllug