The other night I got paged up that one of our VCSA servers had a full partition. After looking into it, it seems that the /storage/archive/ partition was filled up.
When browsing through the partition I quickly noticed that the “vpostgres” folder was the culprit. It was all filled up with over 25K small files that were apparently beeing used by vpostgresql:
root@vcenter [ /storage/archive/vpostgres ]# ls | wc -l
25623
I found a thread on the VMTN forum which I kindly replied on and it seems that this is by design in the VCSA 6.7 U1 release.
The partition /storage/archive partition can get full by design, as it is aimed at storing as much WAL history as possible, and is automatically cleaned up by the archiver service by removing automatically the oldest WAL segments.
https://kb.vmware.com/s/article/57829
The KB also said that this will be fixed in an upcoming release. So stay tuned and dont increase your /storage/archive/ partition when it is 100% full, it will just fill up again in hours!
I double checked if the files were indeed beeing cleaned up on a regular base and this seems to be the case. I ran the command in the below screenshot and waited 15 minutes and ran it again. I noticed the last file had a later timestamp on it. So it checks out!
EDIT on 24-1-2019:
It seems that the mentioned KB has been updated! VMware has released an update int he form of vCenter Server 6.7 Update 1b. If you are having difficulties on your vCenter and want the above issue fixed, please download and install the update.
5 Comments
Don · January 21, 2020 at 11:09 pm
Symptoms seem to be present in 6.7-U2C as well (build 14070457). Thanks for the write-up very helpful and we can confirm our old files are being cleaned up regularly.
Bryan van Eeden · January 22, 2020 at 9:44 am
No problem! But ow boy, it got back again? Just remember that it is full by design. The error message should be muted/not available anymore though.
Rick · January 23, 2020 at 5:44 pm
Thanks for this post. Saved me some worry. I can also confirm that it’s still the case in U2C. I don’t get the alert in vCenter, but it still flags it in vROPS.
Tracy Watson · March 16, 2020 at 10:09 pm
I can confirm that this is still an issue in vCenter 6.7 U3b. Ridiculous.
Bryan van Eeden · March 20, 2020 at 9:55 am
Well, this isn’t actually “fixed” because it isn’t really broken. The mentioned update did fix the Alarm though. That shouldn’t be there anymore.