Built with anycoder

Ubuntu Shell Scripts for Hardened LLM Container

Complete deployment scripts with all critical fixes + Build, Deploy & Runtime security phases

Critical Fixes Applied in v2.1

FIX #1: exec for Signal Handling

Using exec /usr/local/bin/llama-server ensures the LLM server replaces the shell process (PID 1), receiving Docker's SIGTERM directly for proper graceful shutdown.

FIX #2: Build Phase - Vulnerability Scanning

Integrated Trivy scanner in CI/CD pipeline to scan container image layers for CVEs before pushing to registry.

FIX #3: Deploy Phase - Policy Enforcement

Kyverno policies for Kubernetes admission control - enforcing non-root, resource limits, and network isolation.

FIX #4: Runtime Phase - Active Protection

Falco rules for real-time syscall monitoring and anomaly detection with automatic threat response.

Quick Start

# 1. Download and make executable
chmod +x gemma3-*.sh

# 2. Run the setup script (requires sudo)
sudo ./gemma3-setup.sh

# 3. Build with vulnerability scanning
./gemma3-build-scan.sh

# 4. Deploy with policy enforcement (Kubernetes)
kubectl apply -f kyverno-policies.yaml
kubectl apply -f gemma3-deployment.yaml

# 5. Enable runtime protection
./gemma3-runtime-protect.sh

# 6. Verify security posture
./gemma3-verify.sh

Three-Phase Security Architecture

Build Phase

Vulnerability Scanning

Integrated into CI/CD, tools scan container image layers for known CVEs. This prevents vulnerable code from reaching the registry.

Trivy Clair Grype Docker Scout

Deploy Phase

Policy Enforcement

Kubernetes Admission Controllers check manifests against security policies before pods start (non-root, resource limits, network isolation).

Kyverno OPA Gatekeeper Pod Security

Runtime Phase

Active Protection

Real-time monitoring of syscalls, file access, and network activity. Detects and blocks anomalous behavior based on behavioral profiles.

Falco Sysdig Cilium

Signal Handling Fixed

The entrypoint now uses exec to replace the shell with llama-server, ensuring proper SIGTERM delivery for graceful shutdown of GPU resources.

Prerequisites Required

These scripts require Ubuntu 20.04+, Docker 20.10+, and sudo privileges. For Kubernetes features: kubectl, helm, and cluster admin access.

gemma3-setup.sh
Main Setup

Complete system setup script that configures the host, builds the Docker image, and applies all security hardening.

  • Installs Docker and dependencies
  • Creates non-root user (UID 1000)
  • Generates Dockerfile with security
  • Configures seccomp and AppArmor
  • Sets up proper logging to stdout
gemma3-entrypoint.sh
Container Entry

Container entrypoint with exec-based process replacement for proper signal handling and graceful shutdown.

  • exec for PID 1 signal handling
  • Hardware RNG verification
  • Entropy pool monitoring
  • Stdout/stderr logging only
  • Graceful SIGTERM handling
gemma3-build-scan.sh
NEW: Build Phase

CI/CD integration script for vulnerability scanning with Trivy before pushing images to registry.

  • Trivy vulnerability scanner
  • CVE severity thresholds
  • SBOM generation
  • Registry push gates
  • JSON/SARIF reports
kyverno-policies.yaml
NEW: Deploy Phase

Kubernetes admission controller policies for enforcing security requirements at deployment time.

  • Non-root enforcement
  • Resource limits required
  • Read-only root filesystem
  • Host namespace blocking
  • Privileged container denial
gemma3-runtime-protect.sh
NEW: Runtime Phase

Falco-based runtime security with custom rules for LLM container behavioral monitoring.

  • Syscall monitoring
  • File access detection
  • Network anomaly alerts
  • Shell spawn detection
  • Automatic threat response
gemma3-verify.sh
Security Audit

Comprehensive verification script that audits all three security phases and generates a compliance report.

  • Filesystem permissions check
  • Capability verification
  • Seccomp profile validation