Re: Rare runsv logging problem

From: Caleb Spare <cespare_at_gmail.com>
Date: Sat, 26 Jul 2014 10:40:47 -0700

Hi Alex,

This had occurred to me as well, but as you say, svlogd would just be
restarted if the log directory doesn't exist. I've tested this and it
seems to work fine (say, by inserting a sleep at the top of ./run).
Perhaps there is a more particular timing/race that needs to happen to
cause the problem, though.

The reason for making the log directory from ./run, and for making the
log directory owned by user1, is that my application also uses the log
directory (as you mentioned). So I definitely want the directory to be
created before it runs. I suppose I could put the mkdir -p in *both*
./run and ./log/run.

My next ideas for troubleshooting are:

- Do a "deploy"/restart/stop/start loop on a test server to see if I
can induce the condition
- Browse the runit source to see if I can get a better idea of what
could be happening

Thanks for everyone's suggestions so far. Please keep 'em coming!

-Caleb

On Sat, Jul 26, 2014 at 2:44 AM, Alex Efros <powerman_at_powerman.name> wrote:
> Hi!
>
> On Fri, Jul 25, 2014 at 11:16:50PM -0700, Caleb Spare wrote:
>> run:
>>
>> #!/bin/sh
>>
>> mkdir -p /mnt/log/baz
>> chown -R user1 /mnt/log/baz
>> cd /opt/baz/current
>> exec chpst -u user1 ./run_baz 2>&1
>>
>> log/run:
>>
>> #!/bin/sh
>>
>> # The main run script takes care of ensuring the log dir exists.
>> exec svlogd -ttt /mnt/log/baz/
>
> I believe ./run and ./log/run are started simultaneously, so svlogd may
> run before mkdir or between mkdir and chown. I'm not sure this may result
> in having no logger at all as you explained because if svlogd fail it
> should be restarted. Anyway, I think you should move mkdir from ./run to
> ./log/run.
>
> Also it doesn't clear why you change owner of log directory to "user1" -
> usually log directory is owned by user "log" (and svlogd run as user
> "log") while your service run using some other user (this suppose you
> didn't try to access log directory from your application).
>
> So, maybe you should try this:
>
> --- ./run:
> #!/bin/sh
> cd /opt/baz/current
> exec chpst -u user1 ./run_baz 2>&1
>
> --- ./log/run:
> #!/bin/sh
> mkdir -p /mnt/log/baz
> chown -R log /mnt/log/baz
> exec chpst -u log svlogd -ttt /mnt/log/baz/
>
> --
> WBR, Alex.
Received on Sat Jul 26 2014 - 17:40:47 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:18 UTC