@rey I had wondered what happened....
Log files failed to rotate and blew up your storage, or the slightly harder solution of "people keep posting too many cute fox videos and pictures they're taking all my storage!"
wait.... it's gotta be the former, there's no such thing as too many cute fox videos and pictures. :-D
@Accalia so here's what happened
the hourly db backup caused the disk to hit 85% usage
this caused elasticsearch to trip out
this caused all of the ES jobs to fail
each failing job produced a traceback
these tracebacks went into the docker console log for that container
the logs filled up the disk
the disk filling up crashed everything
and i had to get out of bed to fix it
@rey ...... oofh. cascading failures.....
well at least that's sorted out now, right?
@Accalia i hope so!!
The Vulpine Club is a friendly and welcoming community of foxes and their associates, friends, and fans! =^^=