LFCS Linux Foundation Certified System Administrator Study Guide¶
Independent AI-assisted resource — SOURCES + OBJECTIVES CHECKED; HUMAN REVIEW PENDING. Objective coverage, citations, volatility labels, links, and exam-integrity compliance were checked on September 29, 2026. See the sources-and-objectives record. The official LFCS page is authoritative.
Study guide at a glance
Start here
Prerequisites: No formal prerequisite; practical readiness requires repeated command-line administration on fresh systems. Practice persistent kernel/services/jobs/packages and recovery; libvirt, containers and SELinux; dual-stack networking, SSH, filtering/NAT/routes/bridges/bonds/proxies; LVM/filesystems/swap/automount/NFS/NBD; Git, certificates, services/performance/disk troubleshooting; and local/LDAP identities, profiles, limits and ACLs.
Official objective domains
- See the objective map in this guide
The active-reading range uses 90–130 words per minute for technical material. Labs, troubleshooting, note-taking, spaced review, and prerequisite gaps add time; it is not a total preparation promise.
Current baseline: Five weighted, distribution-independent domains on the live LFCS page
Lifecycle watch: No objective replacement or retirement is announced
Official delivery snapshot: Online, remotely proctored, performance-based command-line exam; two hours; intermediate level; certification valid for two years; 12-month eligibility, one retake, and two Killer.sh simulator attempts listed
Prerequisite: None formally; readiness requires repeated administration and recovery without a GUI
How to use this guide¶
LFCS evaluates resulting system state. Practice every task with this loop:
- inspect the host, target, dependencies, current runtime/persistent state and logs;
- state the requested end condition and select the smallest supported change;
- protect access/data, note rollback, and make the change from the command line;
- validate the direct result plus service, network, security and dependent behavior;
- restart, reboot or recreate when persistence matters, then validate again.
Work across a current Debian/Ubuntu-family and RPM-family distribution even though the exam no longer requires choosing a platform in advance. Translate package, network and configuration ownership. Build fresh VMs from snapshots; time complete task sets; keep a short verification checklist. Do not memorize proprietary simulator or exam tasks. Linux Foundation’s included simulator is for environment familiarity and skill diagnosis, not a question bank to reproduce here.
About related items: A
Related item:callout adds prerequisite, operational, architectural, or adjacent context that makes the current topic easier to understand. It is useful supporting knowledge, not a claim that the item appears verbatim in the published exam objectives.
Weighted objective map¶
| Domain | Weight | Performance evidence |
|---|---|---|
| 1. Operations deployment | 25% | Persist kernel/service/job/software state; recover systems; operate libvirt, containers and SELinux |
| 2. Networking | 25% | Configure/troubleshoot dual-stack, time, SSH, filtering/NAT/routes, bridges/bonds and proxy/load balancing |
| 3. Storage | 20% | Build and repair LVM/filesystems/swap/automount/remote and network-block storage; measure performance |
| 4. Essential commands | 20% | Use Git; create/troubleshoot services; diagnose performance, constraints, disk space and TLS certificates |
| 5. Users and groups | 10% | Manage local/LDAP identities, profiles, resource limits and ACLs with effective-access validation |
CURRENT BLUEPRINT: The live page publishes 34 competencies, grouped 8/8/7/6/5. The detailed evidence below expands those public topics without claiming a private task list or exact exam environment. The official language table lists LFCS language options; platform-interface language and task language are separate questions to verify when scheduling.
| Domain | Public competency coverage |
|---|---|
| Operations | Kernel parameters; processes/services; scheduled jobs; packages/repositories; failure recovery; libvirt VMs; containers; SELinux |
| Networking | IPv4/IPv6 and names; time; troubleshooting; SSH; filtering/redirection/NAT; routes; bridges/bonds; proxies/load balancing |
| Storage | LVM; VFS; filesystems; remote filesystems/NBD; swap; automount; performance |
| Essential commands | Git; service configuration; performance; application constraints; disk space; certificates |
| Users/groups | Accounts; profiles; limits; ACLs; LDAP identities |
The two included simulator activations last 36 hours each according to the official page. Access time is not prescribed study time, and simulator task counts are not the real exam’s task count. No simulator or recalled tasks are reproduced here.
1. Operations deployment — 25%¶
Kernel, processes, services and jobs¶
Use sysctl to inspect and change supported kernel parameters. A runtime write and a persistent file under the distribution’s sysctl.d ownership are different; load and verify the intended file and check for later overrides. Kernel command-line, module and bootloader changes have their own persistence. Keep recovery access before boot-affecting work.
Inspect processes with ps, pgrep, pstree, top and /proc; understand PID/parent, user, state, environment, open files/sockets, CPU/memory and exit behavior. Signals request termination/reload/other actions; confirm target before kill. Nice/priority affects CPU scheduling rather than all resource bottlenecks. Jobs started from a shell, a scheduler and a service manager have different session/environment/lifecycle behavior.
With systemd, distinguish unit definition, enabled boot relationship, active runtime, failed state, dependencies and target. Use systemctl status, cat, list-dependencies, is-enabled, is-active, start/stop/restart/reload, enable/disable/mask and journalctl purposefully. daemon-reload rereads unit definitions; it does not reload application configuration. Before changing a remote-access service, validate syntax and retain a second session/console.
Schedule recurring or one-shot tasks with cron/at or systemd timers as appropriate. Define identity, environment/PATH, working directory, command, calendar, persistence after missed run, overlap/locking, output and failure. Validate by an observable result and logs, not only by listing the schedule.
Unit state, scheduling and effective constraints¶
The systemd service manual distinguishes process execution from unit state. A successful Type=oneshot service with RemainAfterExit=yes can be active (exited) with no remaining process. Without that setting it can finish successfully and become inactive. Type=exec confirms the executable was invoked; it does not by itself establish application readiness. Choose readiness evidence appropriate to the service.
Proposed Linux-only state exercise: in a disposable VM with an existing unprivileged lfcs-lab account, review a unit containing the following. Its command deliberately does no work; its purpose is observing unit state. It was not installed or executed in this review.
[Unit]
Description=LFCS synthetic completed-action state
[Service]
Type=oneshot
User=lfcs-lab
ExecStart=/usr/bin/true
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
Check the installed systemd version, executable and account, inspect the unit with systemd-analyze verify, then start only the owned practice unit. Compare ActiveState, SubState, Result and process evidence; stop it and remove only the recorded lab files afterward. Syntax verification alone does not establish service behavior.
Timer documentation limits Persistent=yes catch-up behavior to OnCalendar timers. It does not preserve a separate queue entry for every missed interval. If the target service is already active when the timer elapses, the timer leaves it running instead of launching another instance; a repetitive timer combined with an indefinitely active oneshot can therefore surprise you.
systemctl separates enable/disable from start/stop. Execution settings also belong to the service process: an interactive shell’s limits or profile are not proof of its environment. NoNewPrivileges=yes limits gaining privileges through exec; it does not revoke existing rights or govern an unrelated IPC service. ProtectSystem=strict restricts filesystem writes with documented exceptions and allow-lists; it is not a complete network or secret-isolation policy. Inspect effective process limits and supported settings, then test required and denied operations.
Do not assume ExecStart is an ordinary shell command line. Quoting, expansion and supported prefixes follow systemd’s versioned syntax; use an explicit shell only when the task actually needs shell behavior and its input is controlled. No new online-manual feature is assumed to exist on the exam host.
Packages, failure recovery and boot¶
Identify distribution/release/architecture and repository configuration. Search/install/remove/update/verify packages using the native stack; distinguish installed-file ownership from repository package metadata. Protect signing/trust, dependencies, configuration-file handling and service/reboot requirements. An untrusted repository or Internet-piped privileged installer is a supply-chain risk.
Recover by locating the failure boundary: firmware/device, bootloader, kernel/initramfs/module, root filesystem/mount, systemd target/unit or application. Use console/rescue/emergency access, a known-good kernel or installation media where appropriate. Capture logs and storage identity before repair. A bad fstab, full filesystem, missing initramfs driver, invalid unit dependency and SELinux denial require different evidence.
For filesystem failure, protect data and use filesystem-specific check/repair tools only under supported mounted/unmounted conditions. For package/config failure, compare package verification/history and configuration backup. Restore access narrowly instead of disabling the security mechanism or formatting a device.
Virtual machines, containers and SELinux¶
Libvirt manages hypervisor connections, domains/VM definitions, virtual networks and storage pools/volumes. Use virsh or supported tools to define/start/stop/autostart/inspect, attach resources and diagnose console/network/storage. A running VM needs persistent definition and correct boot, network, storage and resource configuration. Snapshots are not automatically independent backups.
Container engines manage images, registries, containers, networks, volumes and lifecycle. Pull/build from trusted inputs, pin version/digest where appropriate, run non-root with minimum privilege, configure environment/secrets safely, map ports, attach volumes, inspect logs and recreate to prove declarative persistence. Rootless and daemonless designs alter privilege/ownership behavior; know the engine installed on the practiced system.
SELinux mandatory access control uses labels/types, domains and policy in addition to Unix permissions. Inspect mode, file/process contexts and audit denials. Prefer restoring expected labels (restorecon/file-context policy), enabling a narrowly appropriate boolean, or creating supported local policy when genuinely required. chcon may be temporary; disabling enforcement is not a completed repair. Verify after relabel and reboot.
Related item: Effective service state is the intersection of unit configuration, identity/permissions, SELinux, network/listener, dependencies, resource limits and application health.
Persistent definitions and SELinux repair evidence¶
The virsh reference distinguishes definition, running state and autostart. define registers configuration without starting the guest. create can start a transient guest or run an existing persistent guest with a one-time configuration, depending on its identity and existing definition. Autostart is a separate requirement; do not enable it merely because persistence is required. Inspect live and saved XML, storage/network dependencies and the requested restart behavior.
For SELinux, Red Hat’s current guide illustrates repairing expected context and port policy. A file-context mapping records the intended label for a path; a labeling tool applies it to existing files. An immediate chcon change is not equivalent to a persistent mapping. Check Unix access and the actual denial, prefer correct labels and narrowly supported booleans/ports, and do not automatically turn an audit log into broad allow policy. Label changes do not replace content integrity, application configuration or request testing. No VM, container or SELinux policy was changed here.
2. Networking — 25%¶
Addresses, names, routes and time¶
Inspect link/interface/address/route/neighbor with ip; configure IPv4 and IPv6 through the distribution-owned persistent mechanism such as NetworkManager or netplan/systemd-networkd. Correct prefix and gateway matter; avoid overlapping address plans. Validate runtime and persistent state after restarting the network stack or rebooting—without cutting off the only remote session.
Hostname and resolver behavior can involve /etc/hosts, hostname configuration, DNS resolver service and search domains. Use getent, resolvectl, dig/host where available to distinguish application/NSS lookup from a direct DNS query. A name may resolve differently by interface, split DNS, cache or address family.
Set timezone and synchronize system time using the installed NTP client/service. Check clock, source/peer, offset, reachability and persistent enablement. Time affects TLS, Kerberos/tokens, logs, files and distributed debugging. Never “fix” a certificate problem by ignoring time evidence.
Static routes need destination/prefix, next hop or interface, optional metric/table and persistence. Test both directions and source address. Use ping selectively, tracepath/traceroute, ss, packet capture only when authorized, and application requests to localize the layer.
SSH, filtering, NAT and forwarding¶
Configure OpenSSH client keys/options/known hosts and server listeners, authentication, user/group/source restrictions, root access, forwarding and logging. Protect private keys, validate server configuration before reload, retain rollback access and test as the intended non-root user. Host-key verification prevents silent machine impersonation.
Packet filtering evaluates direction/hook, interface, state, source/destination, protocol/port and rule order. Identify whether nftables, firewalld, UFW or another frontend owns persistent policy; do not mix abstractions blindly. Default deny needs explicit management and service allowances. Validate allowed and denied paths for IPv4 and IPv6 plus reboot persistence.
Port redirection/DNAT changes destination; SNAT/masquerade changes source; routing/forwarding must also be enabled and filtered. Trace original and translated tuples and the return path. A rule existing in a table does not prove the kernel forwards, the target listens, or replies route correctly.
Bridges, bonds, reverse proxies and load balancers¶
A bridge connects layer-2 interfaces and is common for VM/container networking. A bond aggregates/redundantly uses physical links according to mode and upstream switch expectations. Configure through the owning network stack, move addresses/routes to the correct logical interface, validate member/carrier/failover and preserve management access.
A reverse proxy accepts client requests and forwards them to upstream services, often terminating TLS and adding headers/policy. A load balancer distributes requests across healthy backends. Configure listener, upstreams, health checks, timeouts, TLS/certificates, forwarded client/protocol headers and logging. Validate direct backend and proxied paths, failure removal and recovery. Do not trust client-supplied forwarding headers unless the proxy boundary is controlled.
Troubleshoot every path: link → address/prefix → neighbor → route → DNS → filtering/NAT → listener/TLS/SSH/proxy → application → return path. ss and service logs often separate “not listening” from “blocked.”
Related item: Control-plane configuration is intent; packet capture, socket state and successful/denied requests are data-plane evidence.
Original route, SSH and proxy diagnosis worksheet¶
In a deliberately simplified single routing table with 0.0.0.0/0, 192.0.2.0/24 and 192.0.2.128/25, destination 192.0.2.150 selects the /25, 192.0.2.20 the /24 and 198.51.100.20 the default. This locally checked model ignores policy rules, marks, VRFs, multipath and reachability. On Linux, ip route get can inspect the selected route for a particular destination/source/context; a plausible route still does not prove delivery or return traffic.
| Symptom | Discriminating evidence | Repair boundary |
|---|---|---|
| SSH changes appear correct but a user is rejected | sshd test modes: -t checks configuration/key sanity; -T with suitable -C parameters shows applicable effective settings |
Test an actual intended login while retaining recovery access; syntax alone is insufficient |
| Backend works but proxy fails | Compare the intended host/SNI, trust, upstream address, timeout and proxy logs | Preserve certificate checks; do not fix a host mismatch by disabling verification |
| NAT rule exists but traffic fails | Confirm forwarding, hook/direction, route, translation and return path | Rule presence is not proof of packet traversal |
| One bonded link fails and capacity falls | Check bond mode, link membership and upstream agreement | Redundancy does not imply one flow can consume the sum of all link rates |
Only the loopback TLS exchange later in this guide was executed. Route, SSH, NAT, proxy, bond and physical-network tests remain proposed for the Linux sandbox.
3. Storage — 20%¶
LVM, filesystems and virtual filesystem¶
Inventory with lsblk, blkid, findmnt, df, du and filesystem/LVM tools. The kernel VFS presents a common file API over specific filesystems. A partition/LV is a block device, a filesystem organizes it, and a mount attaches it to a directory. Identify by UUID/label where stable persistence matters.
LVM maps physical volumes into volume groups and allocates logical volumes. Create/extend/move/remove only after confirming exact devices and data. Growing an LV and growing its filesystem are separate operations; shrinking is filesystem-specific and riskier. Validate free extents, mounted use and backup/recovery before change.
Create, label, mount, persist and troubleshoot supported filesystems. /etc/fstab syntax/options/order can block boot; test with a non-destructive mount validation before reboot. Understand read-only remount, capacity versus inode exhaustion, reserved space, deleted-open files, quotas/permissions and filesystem errors. Use lsof/process evidence before truncating or restarting.
Swap can be a partition/file/LV; configure permission, initialization, activation and persistence, then inspect usage/priority. Swap is pressure capacity, not a cure for memory leak or a substitute for RAM sizing.
Remote filesystems, network block devices and automount¶
NFS-style remote filesystems expose files through a server/export and client mount with identity, permissions, name resolution, route/firewall and availability dependencies. Network block device presents remote blocks that the client treats like a disk, so filesystem ownership/locking/concurrency differs. Confirm whether a service expects shared file semantics or exclusive block ownership.
Automounters mount on access and expire idle mounts, reducing boot coupling. Configure map/source/options, start/enable the service, trigger the path and verify mount plus timeout. A directory existing does not prove the remote resource mounted. Protect against hanging unavailable dependencies and unsafe broad exports.
Measure storage with throughput, latency, IOPS, queue depth/utilization and workload pattern. df/du answer capacity allocation, not device latency. Correlate iostat/vmstat or available tools with process, filesystem and application timing. Cache can distort short tests.
Related item: Redundancy, snapshots and remote mounts solve availability/convenience problems; a recoverable backup additionally requires protected retention, integrity and restore testing.
Resize and capacity decisions before touching a device¶
For growth, establish free backing capacity and expand the containing device before the filesystem as required by the chosen stack. For shrink, never make the containing device smaller than the filesystem’s supported new size. resize2fs documents unmounted shrinking and supported mounted growth for ext filesystems; it does not resize the containing partition itself. Confirm backups, actual mount state and distribution support before a filesystem-specific procedure.
VERIFY CURRENT: The current upstream xfs_growfs manual requires a mounted filesystem for growth and now documents limited shrinking of the last allocation group without removing it, with further geometry restrictions. Do not turn an older “XFS can never shrink” rule, or a newer upstream capability, into an unqualified command for an installed distribution. Verify its kernel, xfsprogs, filesystem geometry and support policy; no shrink or growth was executed here.
Original extent model: with an invented 4 MiB physical extent and 120 free extents, a 513 MiB increase needs 129 extents, nine more than available. A 480 MiB increase uses all 120 and leaves no free-extents margin. This arithmetic does not check metadata, thin-pool/data/snapshot capacity or the target filesystem.
Original performance model: 12,000 operations per second at 4 KiB each is 46.875 MiB/s before other limits. A latency or throughput claim also needs workload, caching, queue depth and measurement duration. For an invented 10 GiB capacity with 2 GiB reserved/unavailable and 7 GiB used, only 1 GiB remains in the simple model; df and du can differ for other reasons such as open-deleted files or mount visibility.
The kernel’s NBD overview describes remote block requests, distinct from NFS file access. Do not let multiple clients write an ordinary non-cluster filesystem merely because a block export is reachable; use the filesystem’s supported coordination model. No disk, LVM, swap, mount, NFS or NBD configuration was modified.
4. Essential commands — 20%¶
Git, files and certificates¶
Use Git status/diff/add/commit/log, branch/switch, merge, fetch/pull/push and remote concepts. Preserve reviewable history, keep secrets out, and resolve conflicts by understanding both changes and retesting. A clean working tree does not prove the deployed service matches the intended commit.
Master safe shell use: quoting and expansion, pipes/redirection, search/filter, archive/compress, permissions/links, editors and local documentation. Treat spaces/newlines, symlinks, mount boundaries, recursive flags and paths beginning with - as hazards. Preview selection before bulk operations.
TLS certificates bind a public key and identity through issuer trust. Inspect subject/SAN, issuer/chain, validity, key match, format/permissions and service configuration. Build a CSR/private key safely, install full chain as required, reload and test name/SNI plus expiry. A browser/client trust error can be wrong time, name, chain, issuer or key—not only expiration.
Original certificate inspection and validation exercise¶
PRACTICAL DEPTH: Use OpenSSL 3.5.x in a new empty disposable directory. Save the script outside that directory and run it from inside. It creates only synthetic practice keys, CSR and certificates; -noenc deliberately leaves those throwaway keys unencrypted. Protect the practice directory using the operating system’s access controls, retain nothing for real authentication and clean up the owned directory after both exercises. A Bash umask is not a claim about native Windows ACL enforcement.
OpenSSL req handles CSR generation and self-signature checking; x509 handles inspection and this small test signer. The issuer explicitly chooses the leaf’s extensions instead of blindly copying a request. The example creates a three-day root and one-day server certificate for a reserved fictional name. No CA is installed into a system/browser store.
#!/usr/bin/env bash
set -euo pipefail
# New empty disposable directory only; never use production keys here.
if [ -n "$(ls -A)" ]; then
printf 'Choose an empty practice directory.\n' >&2
exit 2
fi
umask 077
OPENSSL=${OPENSSL:-openssl}
"$OPENSSL" version
cat > ca.cnf <<'EOF'
[req]
prompt = no
distinguished_name = dn
[dn]
CN = LFCS Practice Root
EOF
cat > leaf.cnf <<'EOF'
[req]
prompt = no
distinguished_name = dn
req_extensions = requested
[dn]
CN = lfcs-practice.invalid
[requested]
subjectAltName = DNS:lfcs-practice.invalid
[issued]
basicConstraints = critical,CA:FALSE
keyUsage = critical,digitalSignature
extendedKeyUsage = serverAuth
subjectAltName = DNS:lfcs-practice.invalid
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
EOF
"$OPENSSL" req -new -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
-noenc -keyout ca.key -out ca.crt -days 3 -sha256 -set_serial 1 \
-config ca.cnf -addext 'basicConstraints=critical,CA:TRUE' \
-addext 'keyUsage=critical,keyCertSign,cRLSign' -addext 'subjectKeyIdentifier=hash'
"$OPENSSL" req -new -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
-noenc -keyout leaf.key -out leaf.csr -sha256 -config leaf.cnf
"$OPENSSL" req -in leaf.csr -verify -noout
"$OPENSSL" x509 -req -in leaf.csr -CA ca.crt -CAkey ca.key \
-set_serial 2 -days 1 -sha256 -extfile leaf.cnf -extensions issued -out leaf.crt
"$OPENSSL" x509 -in leaf.crt -noout -subject -issuer -dates -ext subjectAltName
"$OPENSSL" x509 -in leaf.crt -pubkey -noout > cert-public.pem
"$OPENSSL" pkey -in leaf.key -pubout > key-public.pem
cmp cert-public.pem key-public.pem
"$OPENSSL" verify -CAfile ca.crt -no-CApath -no-CAstore \
-purpose sslserver -verify_hostname lfcs-practice.invalid leaf.crt
"$OPENSSL" x509 -in leaf.crt -checkend 3600 -noout
if "$OPENSSL" verify -CAfile ca.crt -no-CApath -no-CAstore \
-purpose sslserver -verify_hostname wrong.invalid leaf.crt; then
printf 'Expected a hostname failure.\n' >&2
exit 3
fi
# Inspect only public certificate material; retain fixtures for the TLS exercise.
# No CA is installed into a system or browser trust store.
Use verify with an explicit trust anchor, server purpose and expected name. Verification options make those checks distinct: an unqualified chain check does not request every application-purpose check. A matching public-key component proves the selected key/certificate pair agrees; it does not establish hostname, trust or expiry. x509 -checkend checks an expiry window, not the whole validation policy. verify -attime changes only its reference time; do not change the host clock to test expiration.
The first hostname check should pass and the deliberately wrong name should fail. The harness additionally rejected inappropriate client purpose, future-expired/past-not-yet-valid certificates, a corrupted signature, a conflicting SAN despite matching CN and a mismatched private key. PEM/DER conversion preserved the certificate and successful verification. No intermediate chain, revocation policy or production CA lifecycle was tested.
Original loopback TLS probe¶
Save this Python 3.13 example outside the certificate directory and run it with that directory as the current working directory. It connects only to 127.0.0.1 on an OS-selected port. The requested server name controls TLS identity checking; no DNS lookup or hosts-file change is needed. The Python SSL documentation describes the client context’s required trust and hostname verification.
import socket
import ssl
import threading
def probe(cert_file, key_file, ca_file, hostname):
"""One loopback-only handshake and synthetic ping/pong; never alter DNS/trust."""
server_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
server_context.load_cert_chain(cert_file, key_file)
client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
if ca_file is not None:
client_context.load_verify_locations(cafile=ca_file)
# With no CA file, this new context intentionally has no loaded trust anchors.
result = {'client_data': None, 'client_error': None, 'server_data': None,
'server_error': None, 'protocol': None}
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listener:
listener.bind(('127.0.0.1', 0))
listener.listen(1)
listener.settimeout(5)
address = listener.getsockname()
def serve_once():
try:
connection, _ = listener.accept()
with connection:
connection.settimeout(5)
with server_context.wrap_socket(connection, server_side=True) as secured:
data = b''
while len(data) < 4:
chunk = secured.recv(4 - len(data))
if not chunk:
break
data += chunk
result['server_data'] = data.decode('ascii')
if data == b'ping':
secured.sendall(b'pong')
except (OSError, ssl.SSLError) as error:
result['server_error'] = type(error).__name__
worker = threading.Thread(target=serve_once, daemon=True)
worker.start()
try:
with socket.create_connection(address, timeout=5) as raw:
with client_context.wrap_socket(raw, server_hostname=hostname) as secured:
result['protocol'] = secured.version()
secured.sendall(b'ping')
data = b''
while len(data) < 4:
chunk = secured.recv(4 - len(data))
if not chunk:
break
data += chunk
result['client_data'] = data.decode('ascii')
except ssl.SSLCertVerificationError as error:
result['client_error'] = {'code': error.verify_code,
'message': error.verify_message}
finally:
worker.join(timeout=6)
if worker.is_alive():
raise RuntimeError('Practice worker did not stop within its timeout.')
result['listener_closed'] = listener.fileno() == -1
result['worker_stopped'] = not worker.is_alive()
return result
if __name__ == '__main__':
for trust, name in [('ca.crt', 'lfcs-practice.invalid'),
('ca.crt', 'wrong.invalid'),
(None, 'lfcs-practice.invalid')]:
print(probe('leaf.crt', 'leaf.key', trust, name))
Expected outcomes are a verified ping/pong, a rejected wrong hostname and a rejected certificate when the new client context has no trust anchors. Rejected handshakes deliver no application payload. Do not weaken verification to make either negative case pass. This is TLS transport with a tiny synthetic protocol; it is not an HTTP server, reverse proxy or production service.
Executed September 29, 2026: these exact examples and a harness passed 38 checks. The CLI used Git for Windows’ OpenSSL 3.5.7 with MSYS Bash; Python 3.13.14 used its separate OpenSSL 3.0.21 library. There were six real loopback handshake attempts across harness and standalone execution: two successes and four intended verification failures. Workers stopped, listeners closed and all owned keys/certificates/files were removed. No system trust, DNS, host clock, global configuration, external endpoint, Linux service/storage/network/ACL/SELinux, VM or container changed. Full native Linux labs remain proposed.
For Git staging/conflict practice, the LFCA guide’s original exercise provides previously executed local evidence. It does not substitute for checking the actual deployed artifact or for Linux administration practice.
Services, constraints and performance¶
Create a service unit with correct description/dependencies, command, user/group, working directory, environment/config, restart behavior and installation target. Validate application configuration and executable paths/permissions, reload unit definitions, enable/start, inspect status/logs/listener and reboot. Avoid running as root without need.
Application/service constraints include CPU/memory/process/open-file limits, disk capacity/inodes, ports, permissions/ACL/SELinux, environment, library/package version, certificate/time, dependency and cgroup/service sandbox. Determine the effective setting rather than editing a file that the process never reads.
Troubleshoot performance from symptom, scope and baseline. Correlate CPU utilization/run queue, memory pressure/swap, disk latency/queue, network loss/latency, locks and dependency response. top, ps, free, vmstat, iostat, ss, logs and /proc are starting evidence. More CPU will not repair full inodes, a blocked lock or failed DNS.
For disk-space symptoms, compare df blocks/inodes with du, mount points, quotas and deleted-open files. Identify owner/purpose and retention before removing. Log rotation, application lifecycle and capacity alerting are preventive controls; a blind recursive delete is not.
Related item: Fast task completion comes from a practiced inspect/change/verify pattern and local help fluency—not from skipping validation.
5. Users and groups — 10%¶
Local identity databases map names to UID/GID, groups, home and shell. Create/modify/delete users/groups with native tools, choose system versus interactive accounts, manage primary/supplementary groups, passwords/expiry/lock, home skeleton and file ownership. Preserve service/audit/retention needs during removal.
Shell startup and environment files can be personal or system-wide and differ for login, interactive and non-interactive shells. Define PATH/variables/umask deliberately; do not put secrets in world-readable profiles. Confirm the actual shell/session reads the file.
Resource limits restrict processes, open files, memory/CPU or other resources through PAM limits, systemd/cgroups and shell mechanisms. Determine which layer owns the process and verify the effective limit inside it. Raising a limit without finding the exhaustion source can move the failure.
ACLs add named user/group access beyond owner/group/other. Use getfacl/setfacl, understand mask and default directory ACL, and validate as the target user. Unix permissions, ACLs, SELinux, mount options, service sandbox and application policy all affect access.
LDAP provides centralized user/group directory lookup. Configure client URI, base, TLS trust, bind/anonymous design and NSS/PAM integration according to supported tooling; keep local recovery access. Test directory query, name resolution (getent), authentication, group membership, home/shell and offline/failure behavior. DNS, time, certificates and network policy are common dependencies.
ACL decisions and identity versus authentication¶
The POSIX ACL algorithm checks the matching owner first, then a matching named user, then applicable groups and finally other when no earlier identity class matched. A named user with rw- capped by a r-- mask effectively has read only; it does not fall back to a permissive other entry to recover write access. The mask limits named users and group-class entries, not the owner entry. Default ACLs govern inheritance, while an existing file’s access ACL governs current access. Traversal, SELinux, mount and service restrictions still apply. The bit calculation was checked locally; native Linux ACL execution remains deferred.
SSSD’s LDAP troubleshooting guide explains why successful identity retrieval can coexist with failed authentication: the configured identity and authentication paths can have different TLS behavior. Verify appropriate encryption and certificate trust for both; do not disable checks to make login work. Also check the directory schema used for group membership. A successful getent result alone does not prove password authentication, account authorization, home creation, shell access or offline recovery.
Original diagnosis record: capture which identity was returned, which authentication step failed, the applicable provider/configuration, timestamped client/server evidence and one hypothesis. Test as the intended user with a protected local recovery account available; keep bind credentials out of history and logs. No directory, bind or account operation ran in this review.
Integrated scenarios¶
Scenario 1: Service works locally but not through HTTPS¶
Check unit/process/logs, local listener/request, certificate key/name/chain/time, reverse-proxy configuration/upstream health, DNS, firewall/NAT and remote return path. Correct the narrow fault, validate allowed and denied access plus proxy headers, restart/reboot for persistence and record prevention.
Scenario 2: Storage change prevents boot¶
Use console/rescue, capture journal/block/LVM/filesystem/mount evidence, compare fstab identity/options to actual UUIDs and protect data. Temporarily mount/test, correct persistent state, validate application permissions/SELinux, reboot and verify all mounts/services. Never format simply because a filesystem does not mount.
Scenario 3: New LDAP user cannot run a containerized job¶
Verify directory query, NSS/PAM identity/groups, home/profile, ACL/mode/mask, SELinux denial, container engine/rootless ownership, volume labels, resource limits and scheduled-service environment. Fix the owning layer, test as the user, recreate/reboot if relevant, and retain a local administrator path.
Hands-on labs¶
All eight complete Linux labs remain proposed. The executed certificate and loopback TLS examples are narrower evidence. The earlier native Linux runtime blocker remains deferred; MSYS/Windows checks do not validate systemd, Linux networking, filesystems, SELinux or ACLs. Use disposable authorized VMs and a verified inventory of owned targets.
- Host baseline: inventory current Debian/Ubuntu- and RPM-family VMs: release, kernel, packages, processes, service manager, storage, network and identities. Success: explain command/configuration ownership differences and retain a recovery console; no production hosts.
- Operations/recovery: persist one supported kernel setting, schedule an observable job and configure the synthetic service-state example. Introduce one reversible unit or owned mount-configuration fault. Success: distinguish enabled/runtime/readiness state, recover through the console and prove the requested behavior after reboot; restore only the lab changes.
- VMs/containers/MAC: use an authorized virtualization-capable sandbox to compare domain definition, live state and autostart. Recreate a container with external data and investigate an intended SELinux labeling fault. Success: retain required data, repair narrowly and test under enforcement; remove only recorded guest/network/volume objects.
- Network: configure an isolated dual-stack topology with name/time/route/SSH and minimum firewall/NAT policy. Success: prove allowed and denied paths, actual intended SSH access, return traffic and persistence while keeping console access; revert one injected fault at a time.
- Traffic services: where supported, compare bridge/bond behavior and place a TLS reverse proxy before two synthetic backends. Success: validate host/trust/health and client-header boundaries, remove a failed backend and measure restoration; the loopback TLS example alone does not prove this topology.
- Storage: attach clearly identified disposable virtual disks for LVM/filesystem/swap/automount practice, plus scoped NFS/NBD examples. Success: verify backing-device versus filesystem size, mount identity, capacity/inodes/open files and supported recovery without formatting an unknown device. Record version-specific resize constraints; preserve backups and unmount/disconnect only owned targets for cleanup.
- Identity: manage synthetic accounts/groups/profiles/limits/ACLs and a disposable LDAP client/server. Success: test both allowed and denied access, mask effects, effective process limits, identity lookup versus authentication and directory outage recovery; preserve a local administrator path and remove test secrets/accounts safely.
- Timed capstone: from fresh snapshots complete original mixed service, TLS, network, storage, identity and performance requirements. Success: keep a requirement/evidence checklist, negative cases, reboot validation and cleanup inventory. Diagnose a deliberately ambiguous symptom before repair; do not use recalled simulator or exam tasks.
Original knowledge checks¶
- How do runtime and persistent kernel parameters differ?
- What must be checked before signaling a process?
- Distinguish active, enabled, failed and masked systemd states.
- How does daemon-reload differ from service reload?
- Which context must a scheduled job define?
- What makes a repository trusted and maintainable?
- What are the major boot failure boundaries?
- Why is formatting not a troubleshooting step for an unknown mount failure?
- What must libvirt persistence include beyond a running VM?
- Which container state should survive recreation?
- How should an SELinux denial be repaired?
- How do runtime and persistent IP configuration differ?
- Which evidence separates DNS from application failure?
- Why can clock drift cause authentication and TLS failures?
- What defines a static route?
- Which controls belong in secure SSH administration?
- How do filtering, DNAT and SNAT differ?
- Why must firewall policy be tested for IPv6 and reboot?
- Compare bridge and bond.
- What must a reverse-proxy health check prove?
- How do PV, VG and LV relate?
- Why are LV growth and filesystem growth separate?
- Which evidence distinguishes block, inode and deleted-open exhaustion?
- What makes an
fstabedit safe? - How does swap differ from ordinary storage?
- Compare remote filesystem and network block device.
- How do automount and boot-time mount differ operationally?
- Which metrics describe storage performance?
- What should a Git commit exclude?
- Why must a merge conflict be retested?
- Which fields of a TLS certificate/service must align?
- What belongs in a robust custom service unit?
- How do service limits and shell limits interact?
- Why can adding CPU fail to improve a slow service?
- What must safe disk cleanup establish first?
- Which facts define a local user account?
- When are personal and system-wide profiles read?
- How does an ACL mask affect named entries?
- Which layers can deny access despite permissive mode bits?
-
Which dependencies make LDAP login fail even when the directory is running?
-
Can a systemd unit be active when no process remains, and what does that imply for a repetitive timer?
- Which checks does a certificate expiry-window check fail to replace?
- Why can a certificate with the expected CN still fail when a different DNS SAN is present?
- A named ACL user has rw- and the mask is r--. Can a permissive other entry restore write access?
- With 4 MiB extents and 120 free extents, can a 513 MiB increase fit, and what remains unproved by this arithmetic?
- Why does successful LDAP identity lookup not establish successful secure login?
Answers and reasoning¶
- Runtime changes live kernel state; persistent configuration must load in the intended order at boot.
- PID identity/owner, process purpose/state, dependency and whether the requested signal is safe.
- Active is a unit state and can include a completed oneshot with no running process. Enabled configures activation links, failed records failure and masked blocks ordinary activation; each is distinct from application readiness.
- Daemon-reload rereads unit definitions; reload asks the application to reread its configuration.
- User, environment/PATH, directory, command, schedule, overlap, output, failure and persistence behavior.
- Supported source, signed metadata/packages, correct release/architecture, bounded permissions and update ownership.
- Firmware/device, bootloader, kernel/initramfs/module, root filesystem/mount, init/systemd and application.
- It destroys evidence/data and may target the wrong device; identify filesystem, error and recovery path first.
- A persistent domain definition plus correct storage, network, boot configuration and validated resources. Autostart is separate and required only if the requested behavior includes automatic startup.
- Data/configuration that is explicitly externalized to volumes or services; the writable layer should be disposable.
- Inspect audit/context, restore expected label or configure a narrow supported boolean/policy, then validate enforcement.
ipcan alter running state; the distribution network manager’s files/connections recreate it after restart.- Compare address/name and NSS/resolver evidence while preserving intended host/SNI and certificate verification. A raw-IP HTTPS mismatch alone does not establish application failure.
- Certificates, tickets/tokens and log correlation depend on valid synchronized time.
- Destination prefix, next hop/interface, optional metric/table and an owning persistent configuration.
- Strong key/host verification, scoped users/sources, least privilege, secure server settings, logs and rollback access.
- Filter permits/denies; DNAT rewrites destination; SNAT rewrites source. Routing/forwarding and return path still matter.
- Runtime rules may not reload and IPv4-only success can leave an unintended IPv6 path or outage.
- Bridge connects layer-2 segments; bond combines links for redundancy/throughput according to mode/upstream support.
- Real enough backend readiness to accept the intended request, with correct timeout/removal/recovery—not just an open port when insufficient.
- PVs contribute devices to a VG pool; LVs allocate virtual block devices from that pool.
- LVM changes block-device size; the filesystem has its own allocation structures and resize support.
- Compare
dfblocks/inodes,du, quotas and open-but-deleted files fromlsof/process evidence. - Backup current file, confirm UUID/type/options/mount point, test non-destructively and preserve console/recovery.
- Swap supports virtual-memory pressure and has activation/priority; it is not a normal mounted filesystem.
- Remote FS provides shared file semantics; NBD provides remote raw blocks whose filesystem/concurrency the client controls.
- Boot mount couples startup; automount triggers on path access and can expire, with different failure timing.
- Latency, IOPS, throughput, queue/utilization plus workload pattern and process/application timing.
- Secrets, accidental/generated/binary clutter and unrelated changes; history should be small and reviewable.
- Resolution may alter either branch’s intent; tests validate the combined result.
- Private key match, subject/SAN name, issuer/chain/trust, time validity, format/permissions and service listener/SNI.
- Dependencies, command, identity, directory/environment/config, restart/sandbox/limits, logging and install target.
- The effective process layer matters: systemd/cgroup/PAM may override what an interactive shell reports.
- Constraint may be memory, storage, network, lock, name service, dependency or serialized work rather than CPU.
- Exact mount/object, owner/purpose, retention, backup and whether open files/log rotation/process lifecycle explain usage.
- UID, name, primary/supplementary groups, home, shell, password/lock/expiry and owned resources.
- It depends on login/interactive/non-interactive shell and distribution/shell startup order; verify the actual session.
- The mask caps effective permission for named users/groups and owning group entries.
- Path components, owner/mode, ACL, SELinux, mount, service sandbox and application policy.
-
DNS, network/firewall, TLS trust/time, base/filter/bind, NSS/PAM configuration, groups/home/shell and offline behavior.
-
Yes: a successful oneshot with RemainAfterExit can remain active after exit. A timer does not restart an already-active target, so choose the intended lifecycle and evidence.
- Trust/path, expected hostname, application purpose, signature/key relationship and any required revocation policy. Expiry alone is only one condition.
- The expected identity must match the certificate’s applicable name rules; a conflicting DNS SAN is not repaired by a matching CN. Preserve validation and issue/configure the correct certificate.
- No. That matching named-user entry is capped to read by the mask, without falling back to other. Other controls and directory traversal can restrict access further.
- No: it needs 129 extents, nine more than available. The model does not prove metadata/thin-pool/snapshot headroom, filesystem resize support, device identity or recoverability.
- Lookup, authentication, authorization and session setup are separate. TLS/provider/schema, access policy, home/shell and offline behavior must be tested for the intended login.
Source and freshness notes¶
- Scope, distribution independence, assessment, prerequisites, simulator and credential terms: official LFCS page, checked September 29, 2026.
- Reachable training durations/counts are provider metadata checked September 29, 2026; O’Reilly details remain unverified where access was blocked. Access, pricing and catalogs change.
- Distribution networking/package paths, commands, versions, certificates, SELinux, virtualization/container engines and service implementations must be verified on the practiced/current exam environment.
- Objective snapshot SHA-256:
7e81f913990e564ad0238c7842735843375ec30d94f036f88e194dcbfe77cb63. - This guide independently maps public competencies. It does not reproduce official simulator/exam tasks, proprietary labs or recalled items.
Refresh check — October 29, 2026¶
The current LFCS issuer page still describes a two-hour, performance-based command-line exam; the published objective snapshot and delivery contract remain unchanged. The upstream kernel stable index now lists Linux 6.18.59 (October 26) and 6.12.114 (October 21). For administration practice, use current release notes to rehearse package/update planning, kernel-log inspection, reboot scheduling, and post-update service checks, while following the chosen distribution’s supported kernel and backport policy rather than installing upstream kernels solely to match a version number. Linux kernel stable releases.
Places to learn¶
This is not a complete list and is not meant to be consumed in full. Choose one current structured path, spend more time completing and verifying tasks than watching, use the included simulator for environment practice, and close every gap against the live five-domain map.
| Resource | Access | Estimated time | Best use and boundary |
|---|---|---|---|
| Official LFCS page | Public/exam | 3–5 hours | Map current weights, delivery and simulator; recheck before scheduling |
| Official LFCS curriculum path | Public | 30–60 minutes | Select training; its 3–6 month estimate depends on experience |
| Linux System Administration Essentials (LFS207) | Paid | 50–60 hours listed | Official cross-distribution course with labs and assignments |
| Included Killer.sh simulator | Included with exam | 8–14 hours estimated | Two 36-hour activations; rehearse environment, diagnose gaps, then rebuild tasks independently |
| Pluralsight LFCS path | Paid | 40 hours listed | 12 courses, three July 2026 labs and practice exam; two courses updated September 2026, but the summary retains an older six-domain taxonomy |
| KodeKloud LFCS | Paid | 11.25 video hours listed plus 35–70 lab hours estimated | Public course metadata lists eight modules/103 lessons and two mock-exam titles; one networking section is mislabeled Users and Groups, so map topics individually |
| O’Reilly/KodeKloud LFCS course | Paid; automated catalog access blocked | Earlier 11h57 runtime not reverified | Earlier June 2025 edition/coverage details remain unverified; compare the current product before purchase |
VERIFY CURRENT: Public catalogs checked September 29, 2026; paid interiors, provider labs and assessment contents were not accessed. KodeKloud’s public labels are inconsistent in places; its decimal 11.25 video hours is a provider figure, and the lab-hour range here is only a study-planning suggestion. Pluralsight mixes older course dates with refreshed courses/labs, so duration and headline alignment do not prove every current competency. The actual one-page LF curriculum PDF estimates 3–6 months and explicitly states its suggested courses are not prerequisites.
No current MeasureUp or Whizlabs LFCS product was independently verified. Avoid multiple-choice-only preparation for a performance exam and reject recalled tasks.