Page 1 of 2 12 LastLast
Results 1 to 10 of 20

Thread: Need Assistance With Automated Log Rotation (logrotate)

Hybrid View

  1. Need Assistance With Automated Log Rotation (logrotate)

    I have a SLES 11 server that is accepting syslog data from a remote system (thanks to assistance I got on this forum). The data that system is sending to the SLES 11 system amounts to between 5GB and 6GB per hour. The file it is putting the data into, on the SLES 11 system, is /var/log/audit/taudit.log.

    I've created a file in the /etc/cron.hourly directory called syslog and I did a chmod against that file in this fashion:

    (in the /etc/cron.hourly directory)
    chmod 755 syslog

    This is the contents of that file:

    /var/log/audit/taudit.log {
    compress
    dateext
    maxage 30
    rotate 72
    missingok
    notifempty
    size +4096k
    create 640 root root
    sharedscripts
    postrotate
    /etc/init.d/syslog reload
    endscript
    }

    But I still cannot get a logrotation to happen. And I have stopped and restarted the cron service.

  2. #2
    ab NNTP User

    Re: Need Assistance With Automated Log Rotation (logrotate)

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Logrotate conf files are not executable by themselves so placing one in
    /etc/cron.hourly may result in cron trying to run it but there is no
    executable code in there so you're probably just building up errors
    every hour in /var/log/messages as cron fails to run what is not valid.

    If you want to have hourly log rotation you could perhaps do this
    most-easily moving the /etc/cron.daily/logrotate file into
    /etc/cron.hourly which would make the logrotate script (which is
    executable and has valid code in it too) try to run. You would also
    need to move the current 'syslog' file in /etc/cron.hourly to
    /etc/logrotate.d (be sure to not overwrite the file you probably already
    have in there with that same name).

    Another way you could handle this is to just copy your 'syslog' contents
    into /etc/logrotate.d/syslog as a new section along with the other
    sections, and then still set logrotate to run hourly as mentioned above.

    Good luck.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.18 (GNU/Linux)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQIcBAEBAgAGBQJPjcEjAAoJEF+XTK08PnB5aX4QAICFvE5uOC OuQX2Av/Xb59+O
    sDKRer2NvgYTQ1/ADDGRTIojFq/UicumwLnsGH7vgzx4yVsjWooNo/nhSI1NtV3C
    nAStPYFVvRFSwplbMFciD3qvecG2eyUEBD44N1w9+pMrvRFqfk touYsnSvgc4TlW
    JJwXCqlcdtVOkTEfYfejbdyzMXe+KYGdBGxzeiyevlFRvjelEc 8Qhgp/m3ucAJZR
    WRMh4obqMC0hOD4+JSM9/kO+Ex6HYvCfgdqJ9qap2qf0YlU7iWxzcSRoRfOWkLAU
    o8sSaXHaAFp2DYPKjX6gjBmH6aP6Gjij3EZBiL3xD2ZbmNPgKo TSaC0GiCzjTw3I
    2m/lJIrHCVQlBmJHSG6pL2ZWnivxUFGrV8K0Qf3ZUFXpg4IflfYxV up68Jfx3SdG
    5IfqDYqoSgTvPYrkXYrD6fw9VqagnXeRCN7Sl32+xgYaWKXmQj hzAErjcwbxhpfG
    lZ0PivuIAd1TKLhxciiy0GaTOsccAApsrzAw/tnYf23ZffHAVZ4tIbhRH0sE64Df
    p+pYV3qaZ2tTBQTm+0Fox62mbk4R4/RzQikU3vFsI+rXZP9oBi/hOfq2IZ8G3wcA
    /i8rZkWqM/MKZRUTOP5N187lIeXKR/ltt86ymNIKrgGaU5TUJ/rMDcj60Oig3lBz
    pM+ONJ0YY+8w/J/PoKUK
    =bPyZ
    -----END PGP SIGNATURE-----

  3. Re: Need Assistance With Automated Log Rotation (logrotate)

    Well, I could try that and I'm perfectly willing to try it (and I really appreciate the help you're giving me) but I don't want everything to rotate hourly, just that one log file. Is there any way to accomplish this without rotating every log file hourly?

  4. #4
    ab NNTP User

    Re: Need Assistance With Automated Log Rotation (logrotate)

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Sure, a couple of ways. Before getting into that though, everything
    other than what I've already suggested will probably be more work and/or
    less-clean from a maintenance perspective. What is the drawback to
    rotating hourly? If you are concerned about the number of files (24
    hours * 30 days * 12 months... that kind of thing) then you're probably
    forgetting that logrotate does not force a new log file every single
    time it tuns. The logrotate definitions control how often that happens,
    such as when a file gets to be a certain age, or a certain size, or
    something. Tuning in that area is already done and could just be done
    more to prioritize based on hourly stuff for this new section.

    Going back to other solutions...

    First, don't use logrotate but instead put a script in /etc/cron.hourly
    that renames the file and restarts the service (or at least sends a -HUP
    signal) for you. That'd be really easy, but it doesn't use logrotate
    which you may want to use for some reason.

    Second, copy the logrotate script from /etc/cron.daily to
    /etc/cron.hourly and then modify it to ONLY point to one logrotate
    configuration file which happens to be the one that you are going to
    setup for this one syslog file. Now you have two places to maintain
    both the cron/logrotate scripts as well as the logrotate configuration
    files.

    Good luck.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.18 (GNU/Linux)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQIcBAEBAgAGBQJPjwhcAAoJEF+XTK08PnB549MP/A3+2x50TT74c66FTQMpAyWB
    sqI5LdpWh5wBXz+JSqazRA0DMNO8pz1QwPK2mj5amuSjfY3hrp +u/dzoC2xwCfJj
    ki7lUx9y07br3OKoTB664u6w8tsui7pZwW4tC5aXPqsgoPWZfE iGPTYQ0qyH6LNH
    5f7rE/i++qPlPimTAw/JD5WvVuHxK5AL/MaHKB/bnMr/92Qi6lXk+AZvANaco8yZ
    SZIKdh4XQiKUUT0ymULzObdTUhbMHTjMPvVl8clB9Sr2T2sX/1qjaS949SZSt9aZ
    FQRIadLmFSx7p7czVwZ5iCCRUVLUxOn878+/Mi5nt9qzMO53TmIL/f494SeEe7xi
    mxff0bQpij8m399nBQCjmMhxrfpZ94FovxhSJN3am+2Vu3mrc5 1fjTB6sfVE9rkE
    ipwS7fdvI1sDK4I1rmFcZA/b42eBZbMCeGbEUhHEZD0Ude6lD7M8OiNpC1Hdie5r
    eJKV+rpXYYYabW1iz5hFzIDltKpv8UQy5tRoAyQDQ9rA9qtxYc I41v9HD4revRrW
    8747eCB/qI1iw3ehmIixKKfdpSbZDybarKJZ/p0aMA6xYFF32CQ2X8OccuS5TEy8
    kunmTpnLCWK8WjWT8HLsyZGagjHmZ9oGnwZKSon9eQ1UCiMWXS/GvhXzYMjRctzq
    aGTK96g5JQXNLMXHzW6R
    =u+qL
    -----END PGP SIGNATURE-----

  5. Re: Need Assistance With Automated Log Rotation (logrotate)

    I took the suggestion from your first reply and did that (mv the syslog file from cron.hourly to /etc/logrotate.d and move the logrotate file from cron.daily to cron.hourly). But that has produced another problem that I need to solve (a problem you mentioned in your second post); after it rolls this audit file, compressing it and renaming it with the date as part of the name, it doesn't "re-initialize" the file it just rolled. It leaves that file there, which is not what I want. I want that file deleted so that a new one can be started from a new file (a file that starts out as empty).

    Now, I noticed that in that "syslog" file that I have now moved into the /etc/logrotate.d directory, there is a line that instructs that /etc/init.d/syslog reload should be run. I'm thinking that what I need to do is to create a shell script, maybe also putting it into the /etc/init.d directory and in that shell script have the following command:

    service syslog stop
    rm /var/log/audit/taudit.log
    service syslog start

    and then in that syslog file located in /etc/logrotate.d directory, change that line that says /etc/init.d/syslog reload to
    /etc/init.d/syslogt reload and name my shell script located in /etc/init.d syslogt. Would that work?

    See it isn't just a metter of rolling the log file. I also need to delete the old one while initializing a new one. The whole point being to reduce disk space consumption. Otherwise, I'll run out of disk space in just 2 days.

  6. #6
    ab NNTP User

    Re: Need Assistance With Automated Log Rotation (logrotate)

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    That should happen automatically since otherwise nothing about "log
    rotation" is really happening. You should not need any other script
    besides the 'reload' piece that is already there if you are using syslog
    to write your file. Putting another script in /etc/init.d would be the
    wrong way to go since that's where service scripts go and what you're
    proposing is not a service (/usr/bin or /usr/local/bin or something
    would make more sense).

    Why is it not working as you expect? I'm not sure. You could
    potentially use the pre/post scripts to move the file and then reload
    the applicable service (assuming it's syslog, or works like syslog with
    a 'reload' argument) and then rotate your renamed file in case this is
    some issue with the file handle held by the application not letting
    rotation work properly. You could also add something like the following
    to the postrotate script which is probably much simpler/better:

    echo > /var/log/audit/taudit.log

    Good luck.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.18 (GNU/Linux)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQIcBAEBAgAGBQJPjyXcAAoJEF+XTK08PnB5o1AP/RvtGJi+t9JezzOT/Z95mJPA
    eG3QByk1GqJSzOmL6CzWfZubAclFk5S+BACrt2BkYJN/fpw2DNoJtb2ovi8DdK+a
    TYlMbK1hJ75ChNHZ+gn8WdpUO3+0pshgarv/Z4ByOCNy+8Xv4OGBXt3v3oryrrvc
    UogC/PKVOQp/IKJAJUfePIgsnVLCroIeqgy7j/VUz5SUTh1Dj9Z55MASA8JFEMLJ
    cacuJOvpmWkZxWfY/SEiDMRgBoxzby8cPv7pZcfmFkZgRB45YSWkTT/K8EkDgrG3
    O+HUHARNYbLUg7MLoApAfHNj9N+xvf8g8SyVyZTZ6gsVYyalpQ FUTYJuOFnUBHQO
    T4d/nNpPMi4GdDFvdBFwhUWyPh1h5GSPyRhaKIb1/xTx2PpMRYGR/EpL3TcW4uJ8
    m79wfRlvsp8hoLMBiZPeqMip5ui6HmxoSFVOmc8GfZCyHlSh0d PuevvVI3rud5wk
    8fvEA0lqGOnJ0BIfenxebcZIqcH9rrcpthDHr0RaOWjtn2wjIO e9+cM1b5IWMJLc
    WhoJoK8EfzGyVN+cT9nFyfUfQkU9xgZGTKlPhBHSaKK0U7nt4l 4X3WGMlJaGsjb/
    uVroxWPbtoZj7pJwUeE8qF0NAzvyQQgTfhqfDcBzghX14xMjcB 3MamD5fqDqckZC
    VwXJy2fqnVtGsng439rx
    =Kdx2
    -----END PGP SIGNATURE-----

  7. Re: Need Assistance With Automated Log Rotation (logrotate)

    This is what I decided to try; we'll see if it works.

    I left the logrotate file in the /etc/cron.hourly directory.

    I have a file called syslog-t in the /etc/logrotate.d directory. The contents of that file is as below:

    #
    # /etc/logrotate.d/syslog-t
    #
    ###### Description ######
    #
    # File to rotate the /var/log/audit/taudit.log file every hour, compressing it, changing the compressed file's name, and then
    # re-initializing the original file back to an empty file to receive fresh data
    #
    #### End of Description ####

    /var/log/audit/taudit.log {
    compress
    dateext
    maxage 30
    rotate 72
    missingok
    notifempty
    size +4096k
    create 640 root root
    sharedscripts
    postrotate
    /root/tauditinit
    endscript
    }

    In the /root directory, I have a file called tauditinit. The contents of that file is as follows:

    #! /bin/sh
    #
    #
    #
    # /root/tauditinit
    #
    ### BEGIN SCRIPT INFO
    #
    # Description: Re-initialize the fwaudit.log file
    #
    ### END SCRIPT INFO

    service syslog stop
    rm /var/log/audit/taudit.log-????????
    rm /var/log/audit/taudit.log
    service syslog start

    exit

    (I hope that shell script is right)

    Then I did a chmof 755 tauditinit against the /root/tauditinit file. I'm hoping this will work. Please let me know if I've done something incorrectly.

  8. Re: Need Assistance With Automated Log Rotation (logrotate)

    Hi dwoeltje,

    your postrotate script seems to be set to remove uncompressed rotated files("rm /var/log/audit/taudit.log-????????") - do you actually see those? Or is that a simple clean-up measure in case someone uncompressed old logrotates and leaves them in that directory?

    Regards,
    Jens

    Regards,
    Jens

  9. Re: Need Assistance With Automated Log Rotation (logrotate)

    It seems to create two compressed files; one with a .bz2 extension and one without a .bz2 extension. I don't need two copies of the same thing. So I keep the one with the .bz2 extension and delete the other one. Since it appends the date, I have no easy way of determining (in an automated fashion) what it will append (I'm not a programmer, so while someone else might be able to do this, I cannot), so I was left with trying to figure out a way to delete all files that begin with taudit- but not those that end in .bz2. Assuming that the wild card placeholders would work in Linux the same as they do in DOS (* being a wild card for an unlimited amount of anything and ? being a wildcard for anything but only one character of anything) and given that the date appended will always be eight characters, I simply had it delete taudit-????????. That deletes what I don't want and leaves the files with the .bz2 extension.

    But I've run into another problem. Because I'm rotating hourly and there doesn't seem to be an option in logrotate called timeext, the first logrotate for a given day works just fine. But all logrotates that come after that fail because they will have the same name. So, in addition to a postrotate entry, it would seem that I also need a prerotate entry, one that would change the name of the taudit-????????.bz2 file to taudit-????????.<HH:MM>.bz2 (with the time being where <HH:MM> is located). But that creates a problem:

    1. I don't know how to go about getting the time inserted into the file as part of an automated process. Since all other files in the directory would already be renamed taudit-????????.<HH:MM>.bz2, there would only be one file that would match the filename pattern of taudit-????????.bz2, so I would hope that I could simply do something like mv taudit-????????.bz2 taudit-????????.$TIME.bz2 (or whatever I would have to do to get the time inserted in the filename.

  10. Re: Need Assistance With Automated Log Rotation (logrotate)

    Quote Originally Posted by dwoeltje View Post
    It seems to create two compressed files; one with a .bz2 extension and one without a .bz2 extension.
    That's what somehow worries me, because I've never seen something like it. But maybe this has to do with the hourly instead of daily rotation - does this happen for the first rotate on a day, too? If not, I believe to have an explanation for the behaviour:

    - first a day: logrotate does its thing, successfully.
    - any other rotate that day: the compress is tried, but: There already is a file of that name (taudit-yymmdd.bz2), so the compress aborts with an error. logrotate notices and aborts, too: The original message file therefor is left as-is.

    Quote Originally Posted by dwoeltje View Post
    But I've run into another problem. Because I'm rotating hourly and there doesn't seem to be an option in logrotate called timeext, the first logrotate for a given day works just fine. But all logrotates that come after that fail[...]
    Maybe the easier way to handle this would be to move the (freshly created) compressed log file in the post-rotate action:
    Code:
    mv taudit-$(date +%Y%m%d).bz2 taudit-$(date +%Y%m%d%H%M%s).bz2
    If my assumption is correct, then this should handle both problems at once...

    Regards,
    Jens

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •