Troubleshooting Acquisition
This section walks you through troubleshooting acquisitions that are not working as expected.
Depending on the acquisition type you are using, you may need to check different things.
File-based acquisitions
Check the log file is found and readable
The first thing to check is that the log file is found and readable by the CrowdSec service.
The CrowdSec log will show whether the file was found.
Log file locations change by distribution, you can find the default log location outlined here.
- Linux/Freebsd
- Windows
- Kubernetes
grep '/path/to/your/file.log' /var/log/crowdsec.log
Select-String "/path/to/your/file.log" C:\ProgramData\CrowdSec\log\crowdsec.log
kubectl logs -n crowdsec crowdsec-agent-* | grep '/path/to/your/file.log'
Update the commands above to match your log location and the file you are searching for.
time="2024-04-16T11:47:55Z" level=info msg="Adding file /path/to/your/file.log to datasources" type=file
time="2024-04-16T11:54:26Z" level=warning msg="No matching files for pattern /path/to/your/file.log" type=file
The above log message will log the pattern that is set on the source if you are using a globbing pattern. So you may want to alter the filter to match the parent folder for example: grep '/path/to/your/' /var/log/crowdsec.log
Log file is found but not read
If the log file is found but not read, you may want to check the permissions on the file.
This should be highly unlikely as the CrowdSec service runs as root and should be able to read any file. However, if you are running inside a container environment you may need to check the permissions on the file.
If you are not running inside a container environment, you may be hitting the default file-based acquisition behavior, which uses inotify to watch the file. You can disable this by setting poll_without_inotify to true in the acquisition configuration.
filenames:
- /path/to/your/file.log
poll_without_inotify: true
labels:
type: your_type
Log file is read but not parsed
If the log file is read but not parsed, you may want to check the acquisition is correctly configured.
Type label
First, check that the acquisition configuration matches the example shown on the Hub.
For example, if you are using the NGINX Collection, set type to nginx in the acquisition configuration.
filenames:
- /var/log/nginx/*.log
labels:
type: nginx
If you provide the wrong type, the acquisition will not find the correct parser. Refer to the Collection page when setting up a new acquisition; most collections include an example config.
Explain the log line
If you are still having issues parsing log lines, use cscli to explain a line.
tail -n 10 /path/to/your/file.log | cscli explain -f- --type $TYPE -v
Replace $TYPE with the type you have set in the acquisition configuration.
Keep in mind that certain collections are specifically designed to target specific types of log entries and will not parse every log line. For instance, the sshd collection is intended to only parse lines related to failed authentication and not all entries from the sshd log.
If you are still stuck, reach out on Discord or the community forum.