Automated Code Review: Semgrep Custom Rules, AST Pattern Matching, and Building Org-Specific Security Scanners

Why Semgrep, Not Regex

The fundamental problem with grep-based security scanning is that code is a tree, not a string. Semgrep addresses this by operating on the Abstract Syntax Tree. Consider finding SQL injection in Python:

# Grep catches this:
cursor.execute("SELECT * FROM users WHERE id = " + user_id)

# But misses this (same vulnerability, different formatting):
query = "SELECT * FROM users WHERE id = " + user_id
cursor.execute(
 query
)

# And false-positives on this (safe, parameterized):
cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))

Semgrep works on the AST (Abstract Syntax Tree), so it understands code structure, not just text:

# semgrep rule: python-sql-injection.yaml
rules:
 - id: python-sql-injection
 patterns:
 - pattern: cursor.execute($QUERY, ...)
 - pattern-not: cursor.execute("...", (...))
 - metavariable-pattern:
 metavariable: $QUERY
 patterns:
 - pattern: |
 $X + ...
 - pattern: |
 f"..."
 - pattern: |
 "...".format(...)
 - pattern: |
 "..." % ...
 message: >
 SQL query built with string concatenation or formatting.
 Use parameterized queries instead.
 severity: ERROR
 languages: [python]

This matches any cursor.execute() where the query argument is built with concatenation, f-strings, .format(), or % formatting, but explicitly excludes parameterized queries. It works regardless of whitespace, variable naming, or how many lines the code spans.

Writing Your First Custom Rule

Semgrep rules are YAML files with a surprisingly powerful pattern language. The key building blocks:

rules:
 - id: no-hardcoded-secrets
 patterns:
 # Match: any string assignment that looks like a secret
 - pattern: $KEY = "..."
 # Where the variable name suggests it's a secret
 - metavariable-regex:
 metavariable: $KEY
 regex: (?i)(password|secret|api_key|token|private_key)
 # But not when it's empty or a placeholder
 - pattern-not: $KEY = ""
 - pattern-not: $KEY = "CHANGEME"
 - pattern-not: $KEY = "placeholder"
 - pattern-not: $KEY = "xxx"
 message: |
 Hardcoded secret detected in variable '$KEY'.
 Use environment variables or a secrets manager instead.
 severity: WARNING
 languages: [python, javascript, typescript, java, go, ruby]
 metadata:
 category: security
 confidence: MEDIUM
 cwe: CWE-798

Test it:

# Create test file
cat > test_secrets.py << 'EOF'
# ruleid: no-hardcoded-secrets
api_key = "sk_live_4242424242424242"

# ruleid: no-hardcoded-secrets
DB_PASSWORD = "super_secret_123"

# ok: no-hardcoded-secrets
api_key = ""

# ok: no-hardcoded-secrets
password = os.environ.get("DB_PASSWORD")
EOF

# Run the rule
semgrep --config python-secrets.yaml test_secrets.py

Taint Analysis: Following the Data

This is where Semgrep goes beyond pattern matching. Taint mode tracks data flow from sources to sinks:

rules:
 - id: express-xss
 mode: taint
 message: >
 User input flows into HTML response without sanitization.
 This is a Cross-Site Scripting (XSS) vulnerability.
 severity: ERROR
 languages: [javascript, typescript]
 metadata:
 cwe: CWE-79
 owasp: A7:2017

 pattern-sources:
 - patterns:
 - pattern: req.query.$PARAM
 - patterns:
 - pattern: req.body.$PARAM
 - patterns:
 - pattern: req.params.$PARAM

 pattern-sinks:
 - patterns:
 - pattern: res.send($PAYLOAD)
 - patterns:
 - pattern: res.write($PAYLOAD)

 pattern-sanitizers:
 - patterns:
 - pattern: DOMPurify.sanitize(...)
 - patterns:
 - pattern: escape(...)
 - patterns:
 - pattern: encodeURIComponent(...)

This catches:

// DETECTED: tainted data flows from source to sink
app.get('/search', (req, res) => {
 const query = req.query.q;
 res.send(`<h1>Results for: ${query}</h1>`); // XSS!
});

// DETECTED: even through intermediate variables
app.get('/profile', (req, res) => {
 const userId = req.params.id;
 const name = getUserName(userId);
 const html = `<div>${req.query.displayName}</div>`;
 res.send(html); // XSS via req.query.displayName
});

// NOT DETECTED: sanitizer breaks the taint chain
app.get('/safe', (req, res) => {
 const query = DOMPurify.sanitize(req.query.q);
 res.send(`<h1>Results for: ${query}</h1>`); // Safe
});

Building an Org-Specific Rule Set

Beyond security rules, the rule set can expand to organization-specific patterns:

# Rule: Enforce our internal API client usage
rules:
 - id: use-internal-http-client
 patterns:
 - pattern-either:
 - pattern: fetch(...)
 - pattern: axios.get(...)
 - pattern: axios.post(...)
 - pattern: new XMLHttpRequest()
 - pattern-not-inside: |
 // semgrep-ignore: use-internal-http-client
 ...
 paths:
 include:
 - "src/**"
 exclude:
 - "src/lib/http-client/**"
 - "**/*.test.*"
 message: |
 Use our internal HTTP client (`@myorg/http-client`) instead of
 raw fetch/axios. It handles auth, retries, circuit breaking,
 and distributed tracing automatically.
 severity: WARNING
 languages: [javascript, typescript]

 # Rule: Prevent direct database access outside repositories
 - id: no-direct-db-access
 patterns:
 - pattern-either:
 - pattern: prisma.$MODEL.$METHOD(...)
 - pattern: db.query(...)
 - pattern: knex($TABLE)
 - pattern-not-inside: |
 class $REPO {
 ...
 }
 paths:
 include:
 - "src/**"
 exclude:
 - "src/repositories/**"
 - "src/migrations/**"
 message: |
 Direct database access outside the repository layer.
 All database operations should go through repository classes.
 severity: WARNING
 languages: [javascript, typescript]

CI Integration

The GitHub Actions workflow:

name: Semgrep
on: [pull_request]

jobs:
 semgrep:
 runs-on: ubuntu-latest
 container:
 image: semgrep/semgrep
 steps:
 - uses: actions/checkout@v4

 - name: Run Semgrep
 run: |
 semgrep scan \
 --config .semgrep/ \
 --config p/security-audit \
 --config p/owasp-top-ten \
 --error \
 --json \
 --output results.json \
 --sarif \
 --sarif-output results.sarif

 - name: Upload SARIF
 uses: github/codeql-action/upload-sarif@v3
 with:
 sarif_file: results.sarif
 if: always()

 - name: Comment on PR
 if: failure()
 uses: actions/github-script@v7
 with:
 script: |
 const fs = require('fs');
 const results = JSON.parse(fs.readFileSync('results.json'));
 const findings = results.results || [];

 if (findings.length === 0) return;

 let body = '## Semgrep Security Findings\n\n';
 body += `Found ${findings.length} issue(s):\n\n`;

 for (const f of findings.slice(0, 10)) {
 body += `- **${f.check_id}** (${f.extra.severity})\n`;
 body += ` ${f.path}:${f.start.line}\n`;
 body += ` ${f.extra.message}\n\n`;
 }

 github.rest.issues.createComment({
 issue_number: context.issue.number,
 owner: context.repo.owner,
 repo: context.repo.repo,
 body
 });

Results After Six Months

The numbers tell the story:

Before Semgrep:
 - Manual security review time: ~45 min/PR
 - PR turnaround: 2.3 days average
 - Security findings per quarter: ~12 (found in review)
 - Missed findings (found in prod): ~4/quarter

After Semgrep (+ reduced manual review):
 - Automated scan time: ~90 seconds
 - Manual review (complex PRs only): ~15 min
 - PR turnaround: 4 hours average
 - Security findings per quarter: ~35 (caught in CI)
 - Missed findings (found in prod): 0 last quarter

Custom rules written: 47
 - Security patterns: 23
 - API usage enforcement: 12
 - Architecture guardrails: 8
 - Code quality: 4

The biggest win is not finding more bugs, it is freeing the security team to do actual threat modeling and architecture review instead of checking whether parameterized queries are used. The mechanical stuff is what machines are good at.