Hello all,
Long story short.
The issue is that Logstash is duplicating logs after application restarts due to PVCs being mounted with different minor device versions, causing Logstash to mistakenly treat the same log files as different.
Having said that, I’d like to know if it's normal for the 'minor device number' of an NFS 4.0 volume to change when it’s mounted multiple times.
This appears to be a known issue when working with Network File Systems, as outlined in the Elastic documentation I found online, this happens because NFS can present different minor device versions, which Logstash interprets as different file systems, leading to log duplication:
https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-file#_reading_from_remote_network_volumes
If any of you folks have a second opinion on this, I'd love to hear it.
Thank you very much,
Joel.
