Troubleshooting and FAQ

This guide covers common issues, how to diagnose problems, and frequently asked questions.

Symptoms:

• p99 latency spikes
• Slow client responses
• Timeouts

Diagnosis:

1. Check L2 hit rate (persistent mode):

   rate(swytch_redis_l2_hits_total[5m])
   

   High L2 hits mean the working set exceeds memory. Solution: increase --maxmemory.

2. Check eviction rate:

   rate(swytch_redis_evictions_total[5m])
   

   High evictions cause cache churn. Solution: increase --maxmemory.

3. Check disk I/O (persistent mode):

   iostat -x 1
   

   High %util or await indicates disk bottleneck. Solution: use faster storage (NVMe).

4. Check for slow commands:

   swytch redis --debug  # Logs all commands
   

   Look for KEYS *, large LRANGE, or expensive Lua scripts.

5. Check GC pressure:

   rate(go_gc_duration_seconds_sum[5m])
   

   High GC time indicates memory pressure or allocation-heavy workload.

Symptoms:

• Memory usage higher than expected
• OOM kills
• Evictions despite a low key count

Diagnosis:

1. Check actual memory vs keys:

   redis-cli INFO memory
   redis-cli DBSIZE
   

   Calculate expected: keys × (avg_key_size + avg_value_size + 90)

2. Check for large values:

   redis-cli --bigkeys
   

3. Check for memory fragmentation: Go’s allocator may hold onto freed memory. This is normal and not a leak.

4. Check connection count:

   redis-cli INFO clients
   

   Each connection uses ~1KB. 10,000 connections = 10MB overhead.

Symptoms:

• Connection refused errors
• Can’t connect to Swytch

Diagnosis:

1. Is Swytch running?

   ps aux | grep swytch
   systemctl status swytch
   

2. Is it listening on the right address?

   netstat -tlnp | grep 6379
   ss -tlnp | grep 6379
   

   Check --bind setting. 127.0.0.1 only accepts local connections.

3. Firewall blocking?

   iptables -L -n | grep 6379
   

4. At connection limit? Check logs for “max clients reached” messages.

Symptoms:

database is locked by another process

Cause: Another Swytch instance is using the same database file.

Solution:

1. Find and stop the other instance:

   lsof /path/to/redis.db
   

2. If the previous instance crashed, the lock is automatically released.
3. Never run multiple instances against the same --db-path.

Symptoms:

• Data lost after restart
• Keys disappearing

Diagnosis:

1. Is persistent mode enabled? Check the startup logs for:

   using persistent TieredCache storage: /path/to/redis.db
   

   If you see using in-memory CloxCache storage instead, add --persistent --db-path.

2. Check for crash vs graceful shutdown:

   • Graceful shutdown (SIGTERM): All data saved
   • Process crash: No data loss (OS page cache)
   • Power loss: Up to 10ms of writes lost

3. Check disk space:

   df -h /path/to/db
   

   If disk is full, writes fail silently.

4. Using ghost mode? Ghost mode (--ghost) only persists on eviction or shutdown. Crash loses unflushed L1 data.

Symptoms:

• CPU at 100%
• Slow responses

Diagnosis:

1. Enable pprof:

   swytch redis --pprof 6060
   

2. Capture CPU profile:

   go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30
   

3. Common causes:

   • Expensive Lua scripts
   • Large KEYS * or SCAN operations
   • High connection churn
   • Serialization overhead with large values

Symptoms:

• Swytch exits immediately
• Error messages on startup

Common errors:

# All sections
redis-cli INFO

# Specific section
redis-cli INFO memory
redis-cli INFO stats
redis-cli INFO clients
redis-cli INFO persistence

# Memory stats
redis-cli MEMORY STATS

# Find big keys
redis-cli --bigkeys

# Sample random keys
redis-cli RANDOMKEY

# List connected clients
redis-cli CLIENT LIST

# Kill problematic client
redis-cli CLIENT KILL ADDR 192.168.1.100:45678

# Monitor all commands (use briefly, high overhead)
redis-cli MONITOR

using persistent TieredCache storage: /data/redis.db  # Persistent mode enabled
using in-memory CloxCache storage                      # In-memory mode (no persistence)
redis server listening on 127.0.0.1:6379              # Ready to accept connections

Shutting down...                                      # SIGTERM received

Building infrastructure software that enterprises can depend on requires a sustainable business. We’ve chosen a commercial model that lets us focus on quality, support, and long-term development rather than chasing donations or fighting cloud providers over licensing.

Swytch speaks standard Redis and memcached protocols—your data and integrations aren’t locked in. If you need source access for compliance or security review, contact us about our Enterprise license.

Yes. Swytch is designed for production use. Key production features:

• Comprehensive test suite including race detection
• Graceful shutdown with data persistence
• Prometheus metrics for monitoring
• File locking to prevent corruption

Choose Swytch if you need better persistence guarantees and don’t need clustering. Choose Redis if you need horizontal scaling or prefer open source.

No, Swytch is designed for vertical scaling on a single node. For horizontal scaling, use application-level sharding (hash your keys across multiple Swytch instances) or a proxy like Twemproxy.

Writes fail silently—data stays in L1 memory but is not persisted to disk. Swytch does not automatically evict data to free disk space—disk usage is unbounded unlike memory. Monitor disk usage and add storage or delete old data before it fills.

Yes, any Redis client that supports the RESP protocol works. Swytch implements the Redis 8.x protocol. Check the Redis compatibility page for supported commands.

In persistent mode, simply copy the database file:

cp /data/redis.db /backup/redis-$(date +%Y%m%d).db

You can do this while Swytch is running. See Tiered Storage for details.

Yes, with some limitations. EVAL, EVALSHA, and FUNCTION commands work, but cjson and cmsgpack libraries are not available. See Redis compatibility.

The cache needs time to warm up. After migrating from another cache:

1. Lazy loading: Cache populates as requests come in
2. Working set stabilisation: Frequency-based eviction learns your access patterns

Hit rate typically stabilises after 1-2x your average TTL duration.

File an issue at github.com/bottledcode/swytch-/issues with:

1. Swytch version (swytch --version)
2. Operating system and version
3. Steps to reproduce
4. Expected vs actual behavior
5. Relevant logs

For security issues, email security@getswytch.com instead.