Logs

Published on:

Most important commands to remember

  • journalctl -b -n 20 — inspect recent entries from the current boot.
  • journalctl -o json — retain structured journal fields.
  • jq — select fields without guessing positions in plain text.

Commands and flags

Command or option Meaning
-b Restrict the view to the current boot.
-n 20 Return at most twenty recent matching entries.
--no-pager Print directly and exit.
-o json Emit one JSON object per journal entry.
jq '{time_us: …, unit: …, message: …}' Rename and select the timestamp, system unit, and message fields.
| Pass structured output from journalctl to jq.

__REALTIME_TIMESTAMP is a reception timestamp in microseconds since the Unix epoch, commonly encoded as a JSON string. It is not necessarily the instant the application originally created the event.

The concepts that matter

1. A log is a recorded event, not the whole execution

A log entry records something a component chose to emit: a request, a transition, an error, or another event. It is evidence from an observation point, not a complete replay of everything the system did.

Logging level, sampling, buffering, collection, and retention determine what survives. A missing entry can mean the event did not happen, but can also mean it was not recorded or is outside the current view.

2. Structure makes questions easier to ask

Structured logs keep fields such as timestamp, severity, service, request ID, and message separate. That lets tools filter a request ID or aggregate an error code without fragile text-position assumptions.

Field meaning still matters. A timestamp can describe event creation or collector reception. Severity names are conventions, and one service’s warning may have a different operational meaning from another’s. Consistent schemas improve comparison.

A request can produce logs in a proxy, application, and database. A shared request or trace identifier helps associate those entries even when timestamps are close or clocks differ slightly.

Correlation is not causation by itself. Two entries with similar times may be unrelated, and a repeated request ID can represent retries. Read the operation, component, and attempt context before assembling an incident timeline.

4. Useful logging balances context and cost

A message should explain the failed operation and relevant context well enough to guide investigation. Logging every byte can instead increase storage costs, slow searches, and expose credentials or personal data.

Prefer identifiers and bounded diagnostic detail over dumping entire requests or tokens. Retention and access rules should match the information recorded. A successful log write also does not prove that a remote collector indexed the entry or that an alert evaluated it.

One small example

Optional: read journal entries visible to your current account. The commands do not need elevated access simply to demonstrate the format. They create or delete no log records.

journalctl -b -n 20 --no-pager
journalctl -b -n 20 --no-pager -o json | jq '{time_us: .__REALTIME_TIMESTAMP, unit: ._SYSTEMD_UNIT, message: .MESSAGE}'

Compare the human-readable view with selected JSON fields. unit may be null when _SYSTEMD_UNIT is absent; this does not make the entry invalid. Some journal fields can have arrays or other JSON representations, so do not assume every value is a simple string.

The two commands run at different moments and may include different newest entries. Permission restrictions, no recent entries, or a nonpersistent journal can also limit the view. Treat the result as the last visible entries of this boot, not the machine’s complete history.

Keep this idea: Logs are selected event evidence. Their fields, collection path, and visibility limits determine which conclusions they support.