Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View

Categories: default personal
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

default personal
Jason Ish
filestore v2: use fileinfo records as metadata
As fileinfo records are logged to the main eve log, disable
metadata by default. But when enabled, just use the fileinfo

Metadata is stored in a file named:

where the sha256 is the same as the file logged, the seconds
is the unix timestamp in seconds for the fileinfo record,
and the file_id is an atomically incremented integer per
Suricata instance.

This should allow for each occurrence of the same file to have
its own metadata file. But a collision is expected when running
Suricata repeatedly over the same pcap, as that would be the
exact same occurrence of a file.
Jason Ish
file extract: force sha256 even if truncated
Even if a file is truncated, force the SHA256 if force sha256
is set to yes.

The new file store requires the sha256 regardless of the file
state if it is to be logged, as the filename is based on the
Pierre Chifflier
Rust: remove deprecated functions LoggerFlags::get_logged/set_logged
Victor Julien
destate: test cleanups
Jason Ish
util: move SCCreateDirectoryTree to util-path
Renames SCLogCreateDirectoryTree to SCCreateDirectoryTree
and move into a util module for re-use.

Also moves SCMkDir from suricata-common.h to the more
appropriately names util-path.h.

I would have prefered to use util-file for file related options
but that is already used by file store utilities. util-path
is close enough for file related operations.
Jason Ish
configure: check for utime.h and utime()
Victor Julien
logging: unique id's per log direction
For loggers that register once per direction, use unique id's per

Reshuffle id's to keep tx log id's low so we can use u32 for tracking
logged loggers.
Victor Julien
filestore: minor cleanups and warning fixes
Jason Ish
output-json-file: let caller decide if file is stored
Mainly for the filestore module, which may have its own
knowledge of the file being stored before others.
Jason Ish
filestore (old): register global stat in init func
This doesn't need to be registered from suricata.c. And moving
it to the init function makes sure its only registered if
the logger is actually enabled.
Jason Ish
util-error: new error: SC_ERR_CREATE_DIRECTORY
For logging directory create errors.
Jason Ish
create directory: final arg to control full path or prefix
Give SCCreateDirectoryTree a new argument, final. If true the
full path will be created as a directory. If false, the last
component will not be created as a directory (current
Victor Julien
app-layer: use logger bits to avoid looping
Avoid looping in transaction output.

Update app-layer API to store the bits in one step
and retrieve the bits in a single step as well.

Update users of the API.
Eric Leblond
conf: add function to get child with default
Danny Browning
util-time: Add function to convert timespec to epoch millis
Jason Ish
filestore: only allow one filestore to be enabled
There is probably not too much bad about enabling both, but
open file counts can get messy with both enabled. And v1
should be schedule for deprecation soon enough.
Jason Ish
filestore2: warn once for file errors
Track each type of error warning and only log it once. Also create
a new stat, file_store.fs_errors to count each file system type
error (open, rename, unlink).

Also remove exit stats, they are of limited value.
Jason Ish
util-error: define SC_ERR_MAX
Maurizio Abba
print: Escape backslash in PrintRawUriFp
PrintRawUriFp does not properly escape backslash. This causes confusion
between a \ character and an hex-encoded character. PrintRawUriBuffer,
instead, correctly does backslash-encoding.
Adding proper escaping of backslash to PrintRawUriFp.
Victor Julien
detect: move detect cleanup into util func
Maurizio Abba
time: Force init cached_minute_start array
In offline mode, if the starting timestamp is 0 suricata will never
initialize cached_minute_start array. This cause the timestamp to be
ignored when needed (e.g., in fast.log).

This commit will force the initialization of this array.
Pierre Chifflier
NTP: update logger to use new API
Jason Ish
eve/fileinfo: split record creation from writing
Split the building of the fileinfo record from the writing
of the record so the building can be called from other code.
Specifically the new filestore output which uses fileinfo
records as the metadata.
Victor Julien
http: clean up & improve unittests
Jason Ish
doc: document file-store v2
Jason Ish
suricatactl: a new python script for misc. tasks
Use a new directory, Python to host the Suricata python modules.
One entry point is suricatactl, a control script for
miscalleneous tasks. Currently onl filestore pruning
is implemented.
Victor Julien
thresholds: fix issues with host based thresholds
The flow manager thread (that also runs the host cleanup code) would
sometimes free a host before it's thresholds are timed out. This would
lead to misdetection or too many alerts.

This was mostly (only?) visible on slower systems. And was caused by a
mismatch between time concepts of the async flow manager thread and the
packet threads, resulting in the flow manager using a timestamp that
was before the threshold entry creation ts. This would lead to an
integer underflow in the timeout check, leading to a incorrect conclusion
that the threshold entry was timed out.

To address this,  check if the 'check' timestamp is not before the creation
Eric Leblond
af-packet: synchronize flags sizes
They are passed from config to threads so they need to be of the
same size.
Pierre Chifflier
NTP: ensure parser name is not freed after registration
Victor Julien
stream/midstream: be more liberal with window
Use the wscale setting when updating the window, even if it's very
Jason Ish
filestore v2 - initial version
Filestore v2 is starts as a copy of log-filestore with the
following changes.

- NSS is required as file names as based on the SHA256.
- Work/tmp files are stored in a temp. directory, then
  moved into a directory tree where the directory names
  are the first 2 characters of the hex SHA256.
- Removes the need for a waldo file or pid in the filenames.
Jason Ish
suricatasc: don't use find -delete
For when -delete isn't supported by find. Instead use
-print0 with xargs -0.
Victor Julien
detect: put inspect code for MATCH-list into func
Introduce DetectRunInspectRulePacketMatches to inspect the signatures
match list.
Victor Julien
threads: don't crash in slow shutdown
If TmThreadDrainPacketThreads would take more than 60 seconds, the wait
loop that follows it would reach 'timeout' condition immediately. This
would lead to a null ptr deref of 'tv'.

Fix by not counting the TmThreadDrainPacketThreads and also not doing
the null ptr deref in any case.
Jason Ish
SCPathExists - function to see if a path exists
Returns true if path exists, otherwise false.
Jason Ish
create directory: fix strlcpy usage
The final character was being cut off.
Victor Julien
output: add missing dnp3 profiling labels
Victor Julien
detect: move packet hdr inspect into util func
Victor Julien
app-layer: register per proto logger bits
Create a bitmap of the loggers per protocol. This is done at runtime
based on the loggers that are enabled. Take the logger_id for each
logger and store it as a bitmap in the app-layer protcol storage.

Goal is to be able to use it as an expectation later.
Jason Ish
output: introduce init return type
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.

Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.

Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.