[Gllug] Raw Partitions

[Steve Nelson]

On 11/16/05, Peter Grandi <pg_gllug at gllug.for.sabi.co.uk> wrote:
> >>> On Wed, 16 Nov 2005 12:02:55 +0000, Steve Nelson
> >>> <sanelson at gmail.com> said:
>
> sanelson> Hi All, I need to simulate reading and writing to a
> sanelson> raw partition on a test machine before I perform the
> sanelson> same procedure on a live system, which appears to be
> sanelson> crashing when the same happens.
>
> This is not fully meaningful to me -- what do you mean by «raw
> partition»?

My apologies - I should have stated 'raw device'.  A raw device under
linux is clearly different from a raw device under, say, Solaris.  You
may assume that since I posted to a Linux group, and did not specify a
non-Linux OS that I meant Linux.

As I understand it, it is possible to bind a raw device (ie a
character device with major number 162 - check your ls /dev/raw/*
output) to an existing block device - in my case 2 x 100M partitions
on a shared array.  This is consquently used to perform
cache-bypassing  IO operations, and forms part of the design of
Redhat's GFS 6.0

> There are partitions, block
> devices, raw device access wrappers to block devices
> (deperecated), and 'O_DIRECT' (allegedly).

I find it unlikely that rawdevices are deprecated - perhaps you could
explain.  The current situation is a GFS 6.0 cluster, and raw devices,
wrapped as described above, are used for the quorum data.

> Some links on these subjects:
>
>   http://WWW.TLDP.org/HOWTO/SCSI-2.4-HOWTO/rawdev.html
>   http://WWW.LLNL.gov/linux/ucrl-id-144213.html

Thank you, but I am well aware of these links.

> sanelson> I've got hold of sg_dd for the read/writes, but need
> sanelson> to create a raw partition for experimenation.
>
> It is hard for me to see the relationship between 'sg_dd' and
> «raw partition»s...

Well, due to 'bugs' in dd, according the the raw manual, one should
use sg_dd to perform read/write operations on raw partitions.  My aim
was to attempt to perform read/writre operations in a like manner to
those performed when starting clumanager on a cluster node.

> sanelson> I'm not especially keen on repartitioning my test
> sanelson> machine so I can create a raw device. Is it possible
> sanelson> to create a rawdevice 'within' a current partition.
>
> It is impossible either way.

It is clearly possible - I have done it several times.  If by this you
are (somewhat pedantically) arguing over whether these are 'real' raw
devices, or simply wrapper-like bindings, then by all means direct
them to your (most interesting) blog - on this list they are
considered noise.

> Also, the 'raw' command and raw device wrappers are deprecated,
> depending on the kernel release (and did not seem to have much
> of an effect when I tried them, and seem to be little tested).

Ah - ok - they may be unused on 2.6 kernels - I am using 2.4 kernels
at present, and GFS 6.0 relies upon them.  This is a heavily used an
well-supported configuration, so perhaps deprecated is a slightly
strong word.

> The man page 'raw'(8) says «raw is used to bind a Linux raw
> character device to a block device» so first you have to turn
> the file into a block device using 'losetup'(8). But I would be
> rather wary of stacking things this way, because it might now
> have the intended effects, and/or be not reliable...

Yes, I quite agree, and thanks to you and Christian to pointing me to
losetup.  In the end I simply created a new partition for
experimentation.

> sanelson> Or am I barking up completely the wrong tree [ ... ]
>
> Seems to me that is likely the case, because 'sg_dd', and the
> various non-block-device techniques, are a bit subtle and
> mysterious and easy to misunderstand (and not that reliable).

Not quite sure what you mean here.  Either I have misunderstood how
raw devices work, or I haven't.  I am fairly confident that I
understand the idea behind sg_dd - my only question was whether it was
possible to create some kind of virtual device under an already used
partition, and bind a raw device to it.

> So the talk about «sg_dd», «raw partition», «raw device» and so
> on sounds very strange to me.

I would have thought that the (well documented) use of sg_dd and
friends with raw devices was far from strange.  I grant, however, that
my used of the expression 'raw partition' was innaccurate.  Thank you
for correcting it.

> In case it may help, for various futile attempts to bypass the
> Linux page cache or mitigate its performance problems when doing
> large amounts of streaming IO consider reading the 'team' source
> here: <URL:http://WWW.sabi.co.UK/#almostSources>.

Noted.

> As usual, perhaps it would be helpful if you gave some context
> for your question, like the purpose behind the question and the
> configuration of the system (e.g. kernel version etc.).

As usual I gave context - I am, however, not as prolix in my emails as
you, and I do not believe the list would have been especially
interested in the detailed background concerning an Oracle 10 Cluster
with a failing node, which my investigations thus far lead be to
believe is crashing when it attempts to read quorum data from a raw
device, and is thus fenced and rebooted.

S.
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug