Prev: ZFS - Backup configuration
Next: How to use a dvd image to install Solaris 10 on E 250/E 450
From: Zfs.. on 18 Jan 2010 12:00
Folks,

Some worrying behavior with zfs.

While doing some testing we noticed that we where able to import the
same zpool using the -f option to two separate machines at the same
time.

The zpool resides on SAN storage that both servers can see. Importing
the zpool to one machine is no problem, but while imported there we
can also import it to the second machine.. This is not good.

We then decided to be wicked and mounted a filesystem on one side,
wrote a file into it and then on the other side ran a scrub.. Needless
to say, the server that tried to do the scrub crashed.

So, basically, besides the obvious comment of "just don't do that", is
there a way to "lock" a zpool down to a particular machine while it is
imported ?

I think you should be able to do this because if not then there is
potential for some disastrous situations to arrive.

Any thoughts are welcome !
From: ITguy on 18 Jan 2010 12:07
> While doing some testing we noticed that we where able to import the
> same zpool using the -f option to two separate machines at the same
> time.
>
> The zpool resides on SAN storage that both servers can see. Importing
> the zpool to one machine is no problem, but while imported there we
> can also import it to the second machine.. This is not good.
>
> We then decided to be wicked and mounted a filesystem on one side,
> wrote a file into it and then on the other side ran a scrub.. Needless
> to say, the server that tried to do the scrub crashed.
>
> So, basically, besides the obvious comment of "just don't do that", is
> there a way to "lock" a zpool down to a particular machine while it is
> imported ?

Use Sun cluster. Look up HA ZFS
From: Zfs.. on 18 Jan 2010 12:13
On Jan 18, 5:00 pm, "Zfs.." <cian.scrip...(a)gmail.com> wrote:
> Folks,
>
> Some worrying behavior with zfs.
>
> While doing some testing we noticed that we where able to import the
> same zpool using the -f option to two separate machines at the same
> time.
>
> The zpool resides on SAN storage that both servers can see. Importing
> the zpool to one machine is no problem, but while imported there we
> can also import it to the second machine.. This is not good.
>
> We then decided to be wicked and mounted a filesystem on one side,
> wrote a file into it and then on the other side ran a scrub.. Needless
> to say, the server that tried to do the scrub crashed.
>
> So, basically, besides the obvious comment of "just don't do that", is
> there a way to "lock" a zpool down to a particular machine while it is
> imported ?
>
> I think you should be able to do this because if not then there is
> potential for some disastrous situations to arrive.
>
> Any thoughts are welcome !

Just a quick follow up.

It seems that the box that crashed, crashed because of I/O failure

Jan 18 16:49:51 myhost unix: [ID 743715 kern.notice] Pool 'mypool' has
encountered an uncorrectable I/O failure and the failur
e mode property for this pool is set to panic.
Jan 18 16:49:52 myhost unix: [ID 100000 kern.notice]

This would suggest that the first node that imported the disks first,
actually didnt have control over the disks, even though zpool list
listed the pool.

When the zpool scrub was run, it couldn't actually run the scrub as
the disks weren't there and it panicked.. as per the failmode property
of the pool. I'd expect this.

The second box stayed up and has full access to the pool. If I had of
run zpool export mypool from the 1st node, I assume that everything
would of been ok, however is there anything in zfs that can prevent
this ever happening ?

Some sort of lock property would be nice. i.e. zfs set lockhost=on
mypool

I know that we are using the -f option, which is forcing the import to
the second node... however, there could be a situation where using the
-f while the pool is still active on another system can be extremely
dangerous !

Sorry for the long winded post !
From: Zfs.. on 18 Jan 2010 12:14
On Jan 18, 5:07 pm, ITguy <southa...(a)gmail.com> wrote:
> > While doing some testing we noticed that we where able to import the
> > same zpool using the -f option to two separate machines at the same
> > time.
>
> > The zpool resides on SAN storage that both servers can see. Importing
> > the zpool to one machine is no problem, but while imported there we
> > can also import it to the second machine.. This is not good.
>
> > We then decided to be wicked and mounted a filesystem on one side,
> > wrote a file into it and then on the other side ran a scrub.. Needless
> > to say, the server that tried to do the scrub crashed.
>
> > So, basically, besides the obvious comment of "just don't do that", is
> > there a way to "lock" a zpool down to a particular machine while it is
> > imported ?
>
> Use Sun cluster.  Look up HA ZFS

Thanks for the reply but we've just gotten rid of Sun Cluster and
replaced it with VCS and zfs...
From: Darren Dunham on 18 Jan 2010 12:20
On Jan 18, 9:00 am, "Zfs.." <cian.scrip...(a)gmail.com> wrote:

> While doing some testing we noticed that we where able to import the
> same zpool using the -f option to two separate machines at the same
> time.

Given the "-f" option means "force", that doesn't surprise me. You
can do the same thing with Symantec Volume manager and the correct
flags as well.

> The zpool resides on SAN storage that both servers can see. Importing
> the zpool to one machine is no problem, but while imported there we
> can also import it to the second machine.. This is not good.

So don't use "-force". If it doesn't work without -force, you should
be asking yourself why and what is wrong.

> We then decided to be wicked and mounted a filesystem on one side,
> wrote a file into it and then on the other side ran a scrub.. Needless
> to say, the server that tried to do the scrub crashed.

Yes. If the pool is imported simultaneously, it is almost guaranteed
to be corrupted.

> So, basically, besides the obvious comment of "just don't do that", is
> there a way to "lock" a zpool down to a particular machine while it is
> imported ?

It is by default (at least in later versions). But -f lets you
override the lock. There has to be some way to do this because the
host holding the lock can crash.

The first versions of ZFS did not have such a layer of protection and
multiple non-forced imports would succeed.

--
Darren
 |  Next  |  Last
Pages: 1 2 3 4
Prev: ZFS - Backup configuration
Next: How to use a dvd image to install Solaris 10 on E 250/E 450