Routine maintenance - vacuum, analyse and autovacuum

[Iain]

Hi All,

I'm supposed to put together a plan for the routine maintenance of the new
database we are building using v7.4, and I have a question or two.

I'm interested in the auto vacuum daemon, but havn't located any good
information about it so far, I'd like to know if people generally recommend
using it or not.

If we use it, can we stop using nightly vacuums? I'd like to do this if
possible, since the current version (it's a 7.1 db) of the system runs a
number of batch processes at night (large deletes and large inserts), and
finishes with a vacuum analyse. The vacuum takes quite a long time and the
nightly window is gradually getting tighter and tighter.

On the other hand, the system is quite busy during the day, and once the
batch processes have finished the system is idle until about 9am.

The system is used 365 days a year, which doesn't leave much opportunity for
VACUUM FULL or unload reloads.

If I start a VACUUM FULL, is it ok to interrupt it part way through? I mean,
is a partially completed vacuum full worth anything, and is there any danger
in routinely killing it?

Thanks for your help,
regards
Iain

[Bruno Wolff III]

On Thu, Jan 29, 2004 at 18:59:28 +0900,
Iain <***@mst.co.jp> wrote:

I haven't used auto_vacuum, but I seen comments on the lists indicating
that it works OK.

Post by Iain
If I start a VACUUM FULL, is it ok to interrupt it part way through? I mean,
is a partially completed vacuum full worth anything, and is there any danger
in routinely killing it?

With 7.4 you shouldn't have to do VACUUM FULL. If you have FSM set high
enough, then normal VACUUM should suffice. The exception would be if
you ran something that may the DB much larger than normal and you wanted
to shrink the space used by the DB. Then you would need to run a VACUUM FULL.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Tom Lane]

Post by Iain
If we use it, can we stop using nightly vacuums? I'd like to do this if
possible, since the current version (it's a 7.1 db) of the system runs a
number of batch processes at night (large deletes and large inserts), and
finishes with a vacuum analyse. The vacuum takes quite a long time and the
nightly window is gradually getting tighter and tighter.

Just moving from 7.1 to 7.4 should help, since you can start using plain
vacuum (7.1's vacuum is equivalent to vacuum full IIRC).

Post by Iain
If I start a VACUUM FULL, is it ok to interrupt it part way through? I mean,
is a partially completed vacuum full worth anything, and is there any danger
in routinely killing it?

It's crash-safe but you may leave your indexes bigger than before :-(

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

[Iain]

Post by Tom Lane
Just moving from 7.1 to 7.4 should help, since you can start using plain
vacuum (7.1's vacuum is equivalent to vacuum full IIRC).

That old DB server has been running contnuously for 2 years they tell me.
Apparently it is also growing despite nightly vacuums and a stable data set
size (old data is purged dai.ly and monthly). Nightly delete/insert batches
and vacuums are taking longer and longer too. I suspect the vacuum isn't
doing what they think - possibly due to the FSM being at the default, and
possibly due to ghost processes holding old record versions open. It has
been stable nonetheless. Anyway, I'm sure we can do much better in 7.4.

Post by Tom Lane

Post by Iain
If I start a VACUUM FULL, is it ok to interrupt it part way through? I mean,
is a partially completed vacuum full worth anything, and is there any danger
in routinely killing it?

It's crash-safe but you may leave your indexes bigger than before :-(

OK, that's the kind of thing I wanted to know. Basically, partial runs of
vacuum full are not likely to be of any benefit. So, unless you intend to
let it run to completion, best to avoid it.

Given that our system experiences quite heavy usage during the day, and has
a nightly window for batch processing I'm wondering if the best way to go
about this is to give auto vacuum a miss and just do a standard vacuum after
the batch processes have completed. As long as the FSM is big enough to
handle the amount of data changed in 1 day, it would be OK, right?

Thanks for the input, (and Bruno too)
Iain


---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

[Tom Lane]

Post by Iain

Post by Tom Lane
Just moving from 7.1 to 7.4 should help, since you can start using plain
vacuum (7.1's vacuum is equivalent to vacuum full IIRC).

That old DB server has been running contnuously for 2 years they tell me.
Apparently it is also growing despite nightly vacuums and a stable data set
size (old data is purged dai.ly and monthly). Nightly delete/insert batches
and vacuums are taking longer and longer too. I suspect the vacuum isn't
doing what they think - possibly due to the FSM being at the default, and
possibly due to ghost processes holding old record versions open. It has
been stable nonetheless. Anyway, I'm sure we can do much better in 7.4.

Kinda sounds like index bloat to me --- have you done any looking to
determine which files are growing?

Post by Iain
OK, that's the kind of thing I wanted to know. Basically, partial runs of
vacuum full are not likely to be of any benefit. So, unless you intend to
let it run to completion, best to avoid it.

Right.

Post by Iain
Given that our system experiences quite heavy usage during the day, and has
a nightly window for batch processing I'm wondering if the best way to go
about this is to give auto vacuum a miss and just do a standard vacuum after
the batch processes have completed. As long as the FSM is big enough to
handle the amount of data changed in 1 day, it would be OK, right?

Yes, though plain VACUUM is not as much of a drag on performance as
VACUUM FULL (== 7.1 VACUUM). So you might want to experiment and
see if you can get away with it during the day. Also note there is work
afoot to reduce the VACUUM performance drag even further in 7.5.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

[Iain]

Post by Tom Lane
Kinda sounds like index bloat to me --- have you done any looking to
determine which files are growing?

No, I havn't. I don't actually have access to that machine, and there is a
cirtain reluctance to tamper with it. However Ithink that it will need some
attention soon. I'll keepin mind what you said.

regards
iain


---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings