If you’ve worked with Splunk for a little while then you are probably familiar with the existence of the field _time. With Splunk being a time series data store, it makes sense that every event will have a time. Internally, Splunk parses the timestamp from your event and converts it to epoch (seconds since Jan […]
Proving a Negative
I’ve got this Foo Fighters lyric stuck in my head … All my life I’ve been searching for something. Something never comes, never leads to nothing. This seems, relevant, given my focus on search technologies in my career. Today, I’m going to talk about proving a negative. That is, I’m going to talk about searching […]
Splunk and POSIX capabilities
I seem to catch myself talking about this a lot in Slack, so I’m just going to write it all down here and refer people to it. A common issue for Splunk deployments is how to securely deploy the Universal Forwarder. Best practice says “don’t run anything as root that doesn’t need to”, but there’s […]
Splunk 7.2.2 and systemd
Consider this a draft. I’ll update it as I have time, but I’m posting now because it may help someone. Updated 2019-04-07: Some improvements thanks to Red Hat support. I am also trying to collect the knowledge and experience of other SplunkTrust and Splunk community people in order to document this more completely. Many thanks […]
Splunk pass4SymmKey for deployment client -> deployment server
Introduction So you want to secure your Splunk deployment server? There’s a couple of different angles to consider: Are all clients connecting to a given deployment server permitted to do so? Is the client certain that the deployment server they are talking to is the real one and not an impostor? Let’s start at the […]
RHEL 7 UDP metrics into splunk metrics index
We were discussing this on splunk-usergroups slack, and I said I should post it here and vraptor and dawnrise urged me to do so quickly — so here I am. (Thanks vraptor and dawnrise!) First up, a script to use the nstat tool to grab some kernel UDP metrics and write them out in a […]
Back from the brink?
I really gave up on blogging for a long time. “So busy” and all that. I’m trying to get back, lets just call all of that ‘excuses’. So in support of that, a whole bunch of housekeeping on the site. Latest and greatest remote exploits .. err I mean wordpress 😉 SSL by default thanks […]
Nullqueue Sampling
One of the first things the average Splunk administrator has to learn about the hard way is how to send traffic to the Splunk nullQueue. It’s almost a rite of passage — you configure a new data source, somewhat unaware of the tens of thousands of mostly-useless events it produces. It blows out your license […]
Splunking bash history
The history tools built into the bash shell are rather powerful and a great source of information about what has been done to a system. One thing we can do to make these even more useful is add them as a data source in Splunk. While imperfect (see caveats below), this can be helpful in […]
Quick Hit – disabling SSLv3 in Splunk
Update 20141015 – Splunk’s official advisory has been released. Update 20141016 – Changed from a specific TLS1.2 cipher to the generic “TLSv1.2” suite. Hat tip to @techxicologist. If you’ve not seen that SSLv3 is irreparably broken, go read about it, then grab a strong drink and come back. Splunk (as of release 6.1) does not […]