Compare commits

..

5 Commits

Author SHA1 Message Date
Ahrav Dutta 7798005fd5 Update loop. 2023-01-09 16:45:04 -08:00
Ahrav Dutta 32555f5e87 Call each extra arg on its own. 2023-01-09 16:30:46 -08:00
ahrav f3071aa46a Merge branch 'main' into update-entrypoint-script 2023-01-09 15:46:51 -08:00
Ahrav Dutta ea2de122be update comment. 2023-01-09 15:46:41 -08:00
Ahrav Dutta 83e22d8b16 Update entrypoint. 2023-01-09 12:17:33 -08:00
838 changed files with 2154 additions and 6333 deletions
-19
View File
@@ -1,19 +0,0 @@
# Testing
Most testing is handled automatically by our GitHub Actions workflows.
## Local GitHub Action Testing
In some cases you may wish to submit changes to the Trufflehog GitHub Action. Unfortunately GitHub does not provide a 1st-party testing environment for testing actions outside of GitHub Actions.
Fortunately [nektos/act](https://github.com/nektos/act) enables local testing of GitHub Actions.
### Instructions
1. Please follow [the installation instructions](http://https://github.com/nektos/act#installation) for your OS.
2. The first run of `act` will ask you to specify an image. `Medium` should suffice.
3. You'll need to configure a personal-access-token(PAT) with: `repo:status`, `repo_deployment`, and `public_repo` permissions.
4. Set an environment variable named `GITHUB_TOKEN` with the PAT from the previous step as the value: `$ export GITHUB_TOKEN=<your_PAT>`
5. Run the following command from the repository root: `act pull_request -j test -W .github/workflows/secrets.yml -s GITHUB_TOKEN --defaultbranch main`
6. If the job was successful, you should expect to see output from the scanner showing several detected secrets.
7. If you want to omit the context of a pull request event and just test that the action starts successfully, run: `act -j test -W .github/workflows/secrets.yml -s GITHUB_TOKEN --defaultbranch main`
-1
View File
@@ -7,7 +7,6 @@ on:
- v*
branches:
- main
pull_request:
jobs:
test:
-24
View File
@@ -1,24 +0,0 @@
name: Smoke
on:
pull_request:
jobs:
smoke:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v3
with:
go-version: '1.18'
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.head_ref }}
- name: Smoke
run: |
set -e
go run . git https://github.com/dustin-decker/secretsandstuff.git
go run . github --repo https://github.com/dustin-decker/secretsandstuff.git
-6
View File
@@ -1,6 +0,0 @@
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data with TruffleHog.
entry: trufflehog git file://. --since-commit HEAD --only-verified --fail
language: golang
pass_filenames: false
+1 -1
View File
@@ -9,7 +9,7 @@ RUN --mount=type=cache,target=/go/pkg/mod \
GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o trufflehog .
FROM alpine:3.15
RUN apk add --no-cache bash git openssh-client ca-certificates \
RUN apk add --no-cache git ca-certificates \
&& rm -rf /var/cache/apk/* && \
update-ca-certificates
COPY --from=builder /build/trufflehog /usr/bin/trufflehog
+1 -1
View File
@@ -1,6 +1,6 @@
FROM alpine:3.15
RUN apk add --no-cache bash git openssh-client ca-certificates \
RUN apk add --no-cache git ca-certificates \
&& rm -rf /var/cache/apk/* && \
update-ca-certificates
WORKDIR /usr/bin/
+1 -1
View File
@@ -22,7 +22,7 @@ check:
go vet $(shell go list ./... | grep -v /vendor/)
lint:
golangci-lint run --enable bodyclose --out-format=colored-line-number --timeout 10m
golangci-lint run --out-format=colored-line-number --timeout 10m
test-failing:
CGO_ENABLED=0 go test -timeout=5m $(shell go list ./... | grep -v /vendor/) | grep FAIL
+18 -34
View File
@@ -28,7 +28,7 @@ https://join.slack.com/t/trufflehog-community/shared_invite/zt-pw2qbi43-Aa86hkii
![GitHub scanning demo](https://storage.googleapis.com/truffle-demos/non-interactive.svg)
```bash
docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity
docker run -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity
```
## Examples
@@ -36,8 +36,7 @@ docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --or
### Example 1: Scan a repo for only verified secrets
Command:
```bash
```
trufflehog git https://github.com/trufflesecurity/test_keys --only-verified
```
@@ -60,15 +59,14 @@ Timestamp: 2022-06-16 10:17:40 -0700 PDT
### Example 2: Scan a GitHub Org for only verified secrets
```bash
```
trufflehog github --org=trufflesecurity --only-verified
```
### Example 3: Scan a GitHub Repo for only verified keys and get JSON output
Command:
```bash
```
trufflehog git https://github.com/trufflesecurity/test_keys --only-verified --json
```
@@ -79,8 +77,7 @@ Expected output:
```
### Example 4: Scan an S3 bucket for verified keys
```bash
```
trufflehog s3 --bucket=<bucket name> --only-verified
```
@@ -91,21 +88,9 @@ trufflehog s3 --bucket=<bucket name> --only-verified
+ Why is the scan is taking a long time when I scan a GitHub org
+ Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token
+ It says a private key was verified, what does that mean?
+ Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
+ Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
### Example 5: Scan a Github Repo using SSH authentication in docker
```bash
docker run --rm -v "$HOME/.ssh:/root/.ssh:ro" trufflesecurity/trufflehog:latest git ssh://github.com/trufflesecurity/test_keys
```
### Example 6: Scan individual files or directories
```bash
trufflehog filesystem path/to/file1.txt path/to/file2.txt path/to/dir
```
# What's new in v3?
TruffleHog v3 is a complete rewrite in Go with many new powerful features.
@@ -116,14 +101,14 @@ TruffleHog v3 is a complete rewrite in Go with many new powerful features.
## What is credential verification?
For every potential credential that is detected, we've painstakingly implemented programmatic verification against the API that we think it belongs to. Verification eliminates false positives. For example, the [AWS credential detector](pkg/detectors/aws/aws.go) performs a `GetCallerIdentity` API call against the AWS API to verify if an AWS credential is active.
For every potential credential that is detected, we've painstakingly implemented programatic verification against the API that we think it belongs to. Verification eliminates false positives. For example, the [AWS credential detector](pkg/detectors/aws/aws.go) performs a `GetCallerIdentity` API call against the AWS API to verify if an AWS credential is active.
## Installation
Several options:
### 1. Go
```bash
```
git clone https://github.com/trufflesecurity/trufflehog.git
cd trufflehog; go install
@@ -134,12 +119,12 @@ cd trufflehog; go install
### 3. Docker
> Note: Apple M1 hardware users should run with `docker run --rm --platform linux/arm64` for better performance.
> Note: Apple M1 hardware users should run with `docker run --platform linux/arm64` for better performance.
#### **Most users**
```bash
docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys
docker run -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys
```
#### **Apple M1 users**
@@ -148,7 +133,7 @@ The `linux/arm64` image is better to run on the M1 than the amd64 image.
Even better is running the native darwin binary available, but there is no container image for that.
```bash
docker run --rm --platform linux/arm64 -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys
docker run --platform linux/arm64 -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys
```
### 4. Pip (help wanted)
@@ -173,10 +158,10 @@ TruffleHog has a sub-command for each source of data that you may want to scan:
- github
- gitlab
- S3
- filesystem (files and directories)
- filesystem
- syslog
- circleci
- stdin (coming soon)
- file and stdin (coming soon)
Each subcommand can have options that you can see with the `--help` flag provided to the sub command:
@@ -190,7 +175,6 @@ Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--debug Run in debug mode.
--trace Run in trace mode.
--profile Enables profiling and sets a pprof and fgprof server on :18066.
-j, --json Output in JSON format.
--json-legacy Use the pre-v3.0 JSON format. Only works with git, gitlab, and github sources.
--concurrency=10 Number of concurrent workers.
@@ -223,7 +207,7 @@ Exit Codes:
Try scanning an entire GitHub organization with the following:
```bash
docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity
docker run -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity
```
### TruffleHog OSS Github Action
@@ -233,9 +217,9 @@ docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --or
uses: trufflesecurity/trufflehog@main
with:
# Repository path
path:
path:
# Start scanning from here (usually main branch).
base:
base:
# Scan commits until here (usually dev branch).
head: # optional
# Extra args to be passed to the trufflehog cli.
@@ -280,7 +264,7 @@ repos:
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
# entry: bash -c 'docker run -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
language: system
stages: ["commit", "push"]
```
@@ -318,7 +302,7 @@ detectors:
```
```
$ trufflehog filesystem /tmp --config config.yaml --only-verified
» trufflehog filesystem --directory /tmp --config config.yaml --only-verified
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
Found verified result 🐷🔑
+3 -2
View File
@@ -27,9 +27,10 @@ runs:
- git
- file://${{ inputs.path }}
- --since-commit
- ${{ inputs.base }}
- ${{ inputs.base.default }}
- --branch
- ${{ inputs.head }}
- --fail
- --no-update
- ${{ inputs.extra_args }}
- ${{arg in inputs.extra_args.split(' ')}}${{arg}}${{end}}
- ${{ arg }}
+3 -5
View File
@@ -1,6 +1,4 @@
#!/usr/bin/env bash
#!/usr/bin/env ash
# Parse the last argument into an array of extra_args.
mapfile -t extra_args < <(bash -c "for arg in ${*: -1}; do echo \$arg; done")
/usr/bin/trufflehog "${@: 1: $#-1}" "${extra_args[@]}"
# "$@" should expand all args in an 'array' as they were passed in
/usr/bin/trufflehog "$@"
+17 -16
View File
@@ -11,7 +11,7 @@ replace github.com/gitleaks/go-gitdiff => github.com/trufflesecurity/go-gitdiff
require (
cloud.google.com/go/secretmanager v1.10.0
github.com/Azure/go-autorest/autorest/azure/auth v0.5.11
github.com/TheZeroSlave/zapsentry v1.14.0
github.com/TheZeroSlave/zapsentry v1.12.0
github.com/aws/aws-sdk-go v1.44.83
github.com/bill-rich/disk-buffer-reader v0.1.7
github.com/bill-rich/go-syslog v0.0.0-20220413021637-49edb52a574c
@@ -22,7 +22,7 @@ require (
github.com/envoyproxy/protoc-gen-validate v0.9.1
github.com/fatih/color v1.13.0
github.com/felixge/fgprof v0.9.3
github.com/getsentry/sentry-go v0.18.0
github.com/getsentry/sentry-go v0.16.0
github.com/go-errors/errors v1.4.2
github.com/go-git/go-git/v5 v5.5.2
github.com/go-ldap/ldap/v3 v3.4.4
@@ -37,26 +37,28 @@ require (
github.com/gorilla/mux v1.8.0
github.com/hashicorp/go-retryablehttp v0.7.2
github.com/jlaffaye/ftp v0.1.0
github.com/joho/godotenv v1.5.1
github.com/joho/godotenv v1.4.0
github.com/jpillora/overseer v1.1.6
github.com/kylelemons/godebug v1.1.0
github.com/lib/pq v1.10.7
github.com/mattn/go-colorable v0.1.13
github.com/mattn/go-sqlite3 v1.14.16
github.com/mholt/archiver/v4 v4.0.0-alpha.7
github.com/paulbellamy/ratecounter v0.2.0
github.com/pkg/errors v0.9.1
github.com/rabbitmq/amqp091-go v1.7.0
github.com/sergi/go-diff v1.3.1
github.com/stretchr/testify v1.8.2
github.com/rabbitmq/amqp091-go v1.5.0
github.com/sergi/go-diff v1.2.0
github.com/sirupsen/logrus v1.9.0
github.com/stretchr/testify v1.8.1
github.com/tailscale/depaware v0.0.0-20210622194025-720c4b409502
github.com/xanzy/go-gitlab v0.80.2
go.mongodb.org/mongo-driver v1.11.2
github.com/xanzy/go-gitlab v0.77.0
go.mongodb.org/mongo-driver v1.11.1
go.uber.org/zap v1.24.0
golang.org/x/crypto v0.6.0
golang.org/x/crypto v0.5.0
golang.org/x/exp v0.0.0-20221018205818-5c77f4b2bbd7
golang.org/x/oauth2 v0.5.0
golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783
golang.org/x/sync v0.1.0
golang.org/x/text v0.7.0
google.golang.org/genproto v0.0.0-20221201164419-0e50fba7f41c
google.golang.org/protobuf v1.28.1
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/h2non/gock.v1 v1.1.2
@@ -113,7 +115,6 @@ require (
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe // indirect
@@ -137,13 +138,13 @@ require (
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.6.0 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/net v0.5.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af // indirect
golang.org/x/tools v0.3.0 // indirect
google.golang.org/api v0.103.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20221201164419-0e50fba7f41c // indirect
google.golang.org/grpc v1.51.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
+32 -32
View File
@@ -37,8 +37,8 @@ github.com/Microsoft/go-winio v0.5.2 h1:a9IhgEQBCUEk6QCdml9CiJGhAws+YwffDHEMp1VM
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/ProtonMail/go-crypto v0.0.0-20221026131551-cf6655e29de4 h1:ra2OtmuW0AE5csawV4YXMNGNQQXvLRps3z2Z59OPO+I=
github.com/ProtonMail/go-crypto v0.0.0-20221026131551-cf6655e29de4/go.mod h1:UBYPn8k0D56RtnR8RFQMjmh4KrZzWJ5o7Z9SYjossQ8=
github.com/TheZeroSlave/zapsentry v1.14.0 h1:z6H0lIyKVpdXsP5K8J11b4/+V+QCDLDslZmtXsnduMk=
github.com/TheZeroSlave/zapsentry v1.14.0/go.mod h1:0hUR1Et5tSroEYhWOZcxE2iBqcFkuG7jGGlUmOA7XAs=
github.com/TheZeroSlave/zapsentry v1.12.0 h1:hhfwvD7pQbnCIOvAEGM4NXLiaX4b6W6ZWNtL6awSBGc=
github.com/TheZeroSlave/zapsentry v1.12.0/go.mod h1:00uO/VpPrSJG/XigAfTi0F4WMFIw2DmP/IDVUhPBvNw=
github.com/acomagu/bufpipe v1.0.3 h1:fxAGrHZTgQ9w5QqVItgzwj235/uYZYgbXitB+dLupOk=
github.com/acomagu/bufpipe v1.0.3/go.mod h1:mxdxdup/WdsKVreO5GpW4+M/1CE2sMG4jeGJ2sYmHc4=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 h1:JYp7IbQjafoB+tBA3gMyHYHrpOtNuDiK/uB5uXxq5wM=
@@ -101,8 +101,8 @@ github.com/felixge/fgprof v0.9.3/go.mod h1:RdbpDgzqYVh/T9fPELJyV7EYJuHB55UTEULNu
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/getsentry/sentry-go v0.18.0 h1:MtBW5H9QgdcJabtZcuJG80BMOwaBpkRDZkxRkNC1sN0=
github.com/getsentry/sentry-go v0.18.0/go.mod h1:Kgon4Mby+FJ7ZWHFUAZgVaIa8sxHtnRJRLTXZr51aKQ=
github.com/getsentry/sentry-go v0.16.0 h1:owk+S+5XcgJLlGR/3+3s6N4d+uKwqYvh/eS0AIMjPWo=
github.com/getsentry/sentry-go v0.16.0/go.mod h1:ZXCloQLj0pG7mja5NK6NPf2V4A88YJ4pNlc2mOHwh6Y=
github.com/gliderlabs/ssh v0.3.5 h1:OcaySEmAQJgyYcArR+gGGTHCyE7nvhEMTlYY+Dp8CpY=
github.com/gliderlabs/ssh v0.3.5/go.mod h1:8XB4KraRrX39qHhT6yxPsHedjA08I/uBVwj4xC+/+z4=
github.com/go-asn1-ber/asn1-ber v1.5.4 h1:vXT6d/FNDiELJnLb6hGNa309LMsrCoYFvpwHDF0+Y1A=
@@ -220,8 +220,8 @@ github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9Y
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/joho/godotenv v1.4.0 h1:3l4+N6zfMWnkbPEXKng2o2/MR5mSwTrBih4ZEkkz1lg=
github.com/joho/godotenv v1.4.0/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/jpillora/s3 v1.1.4 h1:YCCKDWzb/Ye9EBNd83ATRF/8wPEy0xd43Rezb6u6fzc=
github.com/jpillora/s3 v1.1.4/go.mod h1:yedE603V+crlFi1Kl/5vZJaBu9pUzE9wvKegU/lF2zs=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
@@ -295,13 +295,15 @@ github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rabbitmq/amqp091-go v1.7.0 h1:V5CF5qPem5OGSnEo8BoSbsDGwejg6VUJsKEdneaoTUo=
github.com/rabbitmq/amqp091-go v1.7.0/go.mod h1:wfClAtY0C7bOHxd3GjmF26jEHn+rR/0B3+YV+Vn9/NI=
github.com/rabbitmq/amqp091-go v1.5.0 h1:VouyHPBu1CrKyJVfteGknGOGCzmOz0zcv/tONLkb7rg=
github.com/rabbitmq/amqp091-go v1.5.0/go.mod h1:JsV0ofX5f1nwOGafb8L5rBItt9GyhfQfcJj+oyz0dGg=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/skeema/knownhosts v1.1.0 h1:Wvr9V0MxhjRbl3f9nMnKnFfiWTJmtECJ9Njkea3ysW0=
github.com/skeema/knownhosts v1.1.0/go.mod h1:sKFq3RD6/TKZkSWn8boUbDC7Qkgcv+8XXijpFO6roag=
github.com/smartystreets/assertions v1.0.1 h1:voD4ITNjPL5jjBfgR/r8fPIIBrliWrWHeiJApdr3r4w=
@@ -320,9 +322,8 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/tailscale/depaware v0.0.0-20210622194025-720c4b409502 h1:34icjjmqJ2HPjrSuJYEkdZ+0ItmGQAQ75cRHIiftIyE=
github.com/tailscale/depaware v0.0.0-20210622194025-720c4b409502/go.mod h1:p9lPsd+cx33L3H9nNoecRRxPssFKUwwI50I3pZ0yT+8=
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
@@ -334,8 +335,8 @@ github.com/trufflesecurity/overseer v1.1.7-custom5/go.mod h1:nT9w37AiO1Nop2VhVhN
github.com/ulikunitz/xz v0.5.6/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/ulikunitz/xz v0.5.10 h1:t92gobL9l3HE202wg3rlk19F6X+JOxl9BBrCCMYEYd8=
github.com/ulikunitz/xz v0.5.10/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/xanzy/go-gitlab v0.80.2 h1:CH1Q7NDklqZllox4ICVF4PwlhQGfPtE+w08Jsb74ZX0=
github.com/xanzy/go-gitlab v0.80.2/go.mod h1:DlByVTSXhPsJMYL6+cm8e8fTJjeBmhrXdC/yvkKKt6M=
github.com/xanzy/go-gitlab v0.77.0 h1:UrbGlxkWVCbkpa6Fk6cM8ARh+rLACWemkJnsawT7t98=
github.com/xanzy/go-gitlab v0.77.0/go.mod h1:d/a0vswScO7Agg1CZNz15Ic6SSvBG9vfw8egL99t4kA=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
@@ -351,15 +352,15 @@ github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yusufpapurcu/wmi v1.2.2 h1:KBNDSne4vP5mbSWnJbO+51IMOXJB67QiYCSBrubbPRg=
github.com/yusufpapurcu/wmi v1.2.2/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.mongodb.org/mongo-driver v1.11.2 h1:+1v2rDQUWNcGW7/7E0Jvdz51V38XXxJfhzbV17aNHCw=
go.mongodb.org/mongo-driver v1.11.2/go.mod h1:s7p5vEtfbeR1gYi6pnj3c3/urpbLv2T5Sfd6Rp2HBB8=
go.mongodb.org/mongo-driver v1.11.1 h1:QP0znIRTuL0jf1oBQoAoM0C6ZJfBK4kx0Uumtv1A7w8=
go.mongodb.org/mongo-driver v1.11.1/go.mod h1:s7p5vEtfbeR1gYi6pnj3c3/urpbLv2T5Sfd6Rp2HBB8=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA=
go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ=
go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI=
@@ -376,8 +377,8 @@ golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220826181053-bd7e27e6170d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.6.0 h1:qfktjS5LUO+fFKeJXZ+ikTRijMmljikvG68fpMMruSc=
golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
golang.org/x/crypto v0.5.0 h1:U/0M97KRkSFvyD/3FSmdP5W5swImpNgle/EHFhOsQPE=
golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20221018205818-5c77f4b2bbd7 h1:lhh9BuEmjIBOdOhMIGSQzdW5LbTtFbUV1k9O/Rlq3SE=
golang.org/x/exp v0.0.0-20221018205818-5c77f4b2bbd7/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE=
@@ -410,11 +411,11 @@ golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220826154423-83b083e8dc8b/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.5.0 h1:GyT4nK/YDHSqa1c4753ouYCDajOYKTja9Xb/OHtgvSw=
golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.5.0 h1:HuArIo48skDwlrvM3sEdHXElYslAMsf3KwRkkW4MC4s=
golang.org/x/oauth2 v0.5.0/go.mod h1:9/XBHVqLaWO3/BRHs5jbpYCnOZVjj5V0ndyaAM7KB4I=
golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783 h1:nt+Q6cXKz4MosCSpnbMtqiQ8Oz0pxTef2B4Vca2lvfk=
golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -456,24 +457,23 @@ golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220825204002-c680a09ffe64/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY=
golang.org/x/term v0.4.0 h1:O7UWfv5+A2qiuulQk30kVinPoMtoIPeVaKLEgLpVkvg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4=
golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/text v0.6.0 h1:3XmdazWV+ubf7QgHSTWeykHOci5oeekaGJBLkrkaw4k=
golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af h1:Yx9k8YCG3dvF87UAn2tu2HQLf2dt/eR1bXxpLMWeH+Y=
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+3 -5
View File
@@ -9,8 +9,6 @@ import (
"text/template"
"github.com/go-errors/errors"
"golang.org/x/text/cases"
"golang.org/x/text/language"
"gopkg.in/alecthomas/kingpin.v2"
)
@@ -26,7 +24,7 @@ func main() {
log.SetPrefix("😲 [generate] ")
kingpin.MustParse(app.Parse(os.Args[1:]))
nameTitle = cases.Title(language.AmericanEnglish).String(*name)
nameTitle = strings.Title(*name)
nameLower = strings.ToLower(*name)
nameUpper = strings.ToUpper(*name)
@@ -67,7 +65,7 @@ type templateJob struct {
}
func mustWriteTemplates(jobs []templateJob) {
log.Printf("Generating %s %s\n", cases.Title(language.AmericanEnglish).String(*kind), nameTitle)
log.Printf("Generating %s %s\n", strings.Title(*kind), nameTitle)
// Make the folder.
log.Printf("Creating folder %s\n", folderPath())
@@ -86,7 +84,7 @@ func mustWriteTemplates(jobs []templateJob) {
for _, rplString := range job.ReplaceString {
tmplRaw = strings.ReplaceAll(tmplRaw, strings.ToLower(rplString), "<<.NameLower>>")
tmplRaw = strings.ReplaceAll(tmplRaw, cases.Title(language.AmericanEnglish).String(rplString), "<<.NameTitle>>")
tmplRaw = strings.ReplaceAll(tmplRaw, strings.Title(rplString), "<<.NameTitle>>")
tmplRaw = strings.ReplaceAll(tmplRaw, strings.ToUpper(rplString), "<<.NameUpper>>")
}
+18 -37
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/paulbellamy/ratecounter"
log "github.com/sirupsen/logrus"
"golang.org/x/sync/semaphore"
"gopkg.in/alecthomas/kingpin.v2"
@@ -18,7 +19,6 @@ import (
"github.com/trufflesecurity/trufflehog/v3/pkg/decoders"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors"
"github.com/trufflesecurity/trufflehog/v3/pkg/engine"
"github.com/trufflesecurity/trufflehog/v3/pkg/log"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/source_metadatapb"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/sourcespb"
"github.com/trufflesecurity/trufflehog/v3/pkg/sources"
@@ -42,20 +42,6 @@ var (
)
func main() {
// setup logger
logger, flush := log.New("trufflehog", log.WithConsoleSink(os.Stderr))
// make it the default logger for contexts
context.SetDefaultLogger(logger)
defer func() { _ = flush() }()
logFatal := func(err error, message string, keyAndVals ...any) {
logger.Error(err, message, keyAndVals...)
if err != nil {
os.Exit(1)
return
}
os.Exit(0)
}
ctx, cancel := context.WithTimeout(context.Background(), time.Hour*2)
var cancelOnce sync.Once
defer cancelOnce.Do(cancel)
@@ -82,19 +68,19 @@ func main() {
} else {
_, ok := allScanners[input]
if !ok {
logFatal(fmt.Errorf("invalid input"), "could not find scanner by that name")
log.Fatal("could not find scanner by that name")
}
selectedScanners[input] = allScanners[input]
}
if len(selectedScanners) == 0 {
logFatal(fmt.Errorf("invalid input"), "no detectors selected")
log.Fatal("no detectors selected")
}
for _, excluded := range *scanCmdExclude {
delete(selectedScanners, excluded)
}
logger.Info("loaded secret detectors", "count", len(selectedScanners)+3)
log.Infof("loaded %d secret detectors", len(selectedScanners)+3)
var wgScanners sync.WaitGroup
@@ -106,7 +92,7 @@ func main() {
time.Sleep(60 * time.Second)
counter.Incr(int64(chunkCounter - prev))
prev = chunkCounter
logger.Info("chunk scan rate per second", "rate", counter.Rate()/60)
log.Infof("chunk scan rate: %d/sec", counter.Rate()/60)
}
}()
@@ -136,7 +122,7 @@ func main() {
res, err := scanner.FromData(ctx, *scanVerify, decoded.Data)
if err != nil {
logFatal(err, "error scanning chunk")
log.Fatal(err)
}
if len(res) > 0 {
if resCounter[name] == nil {
@@ -145,19 +131,16 @@ func main() {
}
atomic.AddUint64(resCounter[name], uint64(len(res)))
if *scanThreshold != 0 && int(*resCounter[name]) > *scanThreshold {
logger.Error(
fmt.Errorf("exceeded result threshold"), "snifftest failed",
"scanner", name, "threshold", *scanThreshold,
)
log.WithField("scanner", name).Errorf("exceeded result threshold of %d", *scanThreshold)
failed = true
os.Exit(1)
}
if *scanPrintRes {
for _, r := range res {
logger := logger.WithValues("secret", name, "meta", chunk.SourceMetadata, "result", string(r.Raw))
logger := log.WithField("secret", name).WithField("meta", chunk.SourceMetadata).WithField("result", string(r.Raw))
if *scanPrintChunkRes {
logger = logger.WithValues("chunk", string(decoded.Data))
logger = logger.WithField("chunk", string(decoded.Data))
}
logger.Info("result")
}
@@ -173,20 +156,18 @@ func main() {
}
for _, repo := range strings.Split(*scanCmdRepo, ",") {
if err := sem.Acquire(ctx, 1); err != nil {
logFatal(err, "timed out waiting for semaphore")
}
sem.Acquire(ctx, 1)
wgChunkers.Add(1)
go func(r string) {
defer sem.Release(1)
defer wgChunkers.Done()
logger.Info("cloning repo", "repo", r)
log.Infof("cloning %s", r)
path, repo, err := git.CloneRepoUsingUnauthenticated(ctx, r)
if err != nil {
logFatal(err, "error cloning repo", "repo", r)
log.Fatal(err)
}
logger.Info("cloned repo", "repo", r)
log.Infof("cloned %s", r)
s := git.NewGit(sourcespb.SourceType_SOURCE_TYPE_GIT, 0, 0, "snifftest", false, runtime.NumCPU(),
func(file, email, commit, timestamp, repository string, line int64) *source_metadatapb.MetaData {
@@ -203,12 +184,12 @@ func main() {
}
})
logger.Info("scanning repo", "repo", r)
log.Infof("scanning %s", r)
err = s.ScanRepo(ctx, repo, path, git.NewScanOptions(), chunksChan)
if err != nil {
logFatal(err, "error scanning repo")
log.Fatal(err)
}
logger.Info("scanned repo", "repo", r)
log.Infof("scanned %s", r)
defer os.RemoveAll(path)
}(repo)
}
@@ -220,9 +201,9 @@ func main() {
wgScanners.Wait()
logger.Info("completed snifftest", "chunks", chunkCounter)
log.WithField("chunks", chunkCounter).Info("completed")
for scanner, resultsCount := range resCounter {
logger.Info(scanner, "results", *resultsCount)
log.WithField("results", *resultsCount).Info(scanner)
}
if failed {
+107 -195
View File
@@ -12,21 +12,18 @@ import (
"time"
"github.com/felixge/fgprof"
"github.com/go-logr/logr"
"github.com/gorilla/mux"
"github.com/jpillora/overseer"
"github.com/sirupsen/logrus"
"gopkg.in/alecthomas/kingpin.v2"
"github.com/trufflesecurity/trufflehog/v3/pkg/common"
"github.com/trufflesecurity/trufflehog/v3/pkg/config"
"github.com/trufflesecurity/trufflehog/v3/pkg/context"
"github.com/trufflesecurity/trufflehog/v3/pkg/decoders"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors"
"github.com/trufflesecurity/trufflehog/v3/pkg/engine"
"github.com/trufflesecurity/trufflehog/v3/pkg/handlers"
"github.com/trufflesecurity/trufflehog/v3/pkg/log"
"github.com/trufflesecurity/trufflehog/v3/pkg/output"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
"github.com/trufflesecurity/trufflehog/v3/pkg/sources"
"github.com/trufflesecurity/trufflehog/v3/pkg/sources/git"
"github.com/trufflesecurity/trufflehog/v3/pkg/updater"
@@ -38,7 +35,6 @@ var (
cmd string
debug = cli.Flag("debug", "Run in debug mode.").Bool()
trace = cli.Flag("trace", "Run in trace mode.").Bool()
profile = cli.Flag("profile", "Enables profiling and sets a pprof and fgprof server on :18066.").Bool()
jsonOut = cli.Flag("json", "Output in JSON format.").Short('j').Bool()
jsonLegacy = cli.Flag("json-legacy", "Use the pre-v3.0 JSON format. Only works with git, gitlab, and github sources.").Bool()
concurrency = cli.Flag("concurrency", "Number of concurrent workers.").Default(strconv.Itoa(runtime.NumCPU())).Int()
@@ -50,11 +46,6 @@ var (
printAvgDetectorTime = cli.Flag("print-avg-detector-time", "Print the average time spent on each detector.").Bool()
noUpdate = cli.Flag("no-update", "Don't check for updates.").Bool()
fail = cli.Flag("fail", "Exit with code 183 if results are found.").Bool()
archiveMaxSize = cli.Flag("archive-max-size", "Maximum size of archive to scan. (Byte units eg. 512B, 2KB, 4MB)").Bytes()
archiveMaxDepth = cli.Flag("archive-max-depth", "Maximum depth of archive to scan.").Int()
archiveTimeout = cli.Flag("archive-timeout", "Maximum time to spend extracting an archive.").Duration()
includeDetectors = cli.Flag("include-detectors", "Comma separated list of detector types to include. Protobuf name or IDs may be used, as well as ranges.").Default("all").String()
excludeDetectors = cli.Flag("exclude-detectors", "Comma separated list of detector types to exclude. Protobuf name or IDs may be used, as well as ranges. IDs defined here take precedence over the include list.").String()
gitScan = cli.Command("git", "Find credentials in git repositories.")
gitScanURI = gitScan.Arg("uri", "Git repository URL. https://, file://, or ssh:// schema expected.").Required().String()
@@ -67,17 +58,15 @@ var (
_ = gitScan.Flag("entropy", "No-op flag for backwards compat.").Bool()
_ = gitScan.Flag("regex", "No-op flag for backwards compat.").Bool()
githubScan = cli.Command("github", "Find credentials in GitHub repositories.")
githubScanEndpoint = githubScan.Flag("endpoint", "GitHub endpoint.").Default("https://api.github.com").String()
githubScanRepos = githubScan.Flag("repo", `GitHub repository to scan. You can repeat this flag. Example: "https://github.com/dustin-decker/secretsandstuff"`).Strings()
githubScanOrgs = githubScan.Flag("org", `GitHub organization to scan. You can repeat this flag. Example: "trufflesecurity"`).Strings()
githubScanToken = githubScan.Flag("token", "GitHub token. Can be provided with environment variable GITHUB_TOKEN.").Envar("GITHUB_TOKEN").String()
githubIncludeForks = githubScan.Flag("include-forks", "Include forks in scan.").Bool()
githubIncludeMembers = githubScan.Flag("include-members", "Include organization member repositories in scan.").Bool()
githubIncludeRepos = githubScan.Flag("include-repos", `Repositories to include in an org scan. This can also be a glob pattern. You can repeat this flag. Must use Github repo full name. Example: "trufflesecurity/trufflehog", "trufflesecurity/t*"`).Strings()
githubExcludeRepos = githubScan.Flag("exclude-repos", `Repositories to exclude in an org scan. This can also be a glob pattern. You can repeat this flag. Must use Github repo full name. Example: "trufflesecurity/driftwood", "trufflesecurity/d*"`).Strings()
githubScanIncludePaths = githubScan.Flag("include-paths", "Path to file with newline separated regexes for files to include in scan.").Short('i').String()
githubScanExcludePaths = githubScan.Flag("exclude-paths", "Path to file with newline separated regexes for files to exclude in scan.").Short('x').String()
githubScan = cli.Command("github", "Find credentials in GitHub repositories.")
githubScanEndpoint = githubScan.Flag("endpoint", "GitHub endpoint.").Default("https://api.github.com").String()
githubScanRepos = githubScan.Flag("repo", `GitHub repository to scan. You can repeat this flag. Example: "https://github.com/dustin-decker/secretsandstuff"`).Strings()
githubScanOrgs = githubScan.Flag("org", `GitHub organization to scan. You can repeat this flag. Example: "trufflesecurity"`).Strings()
githubScanToken = githubScan.Flag("token", "GitHub token. Can be provided with environment variable GITHUB_TOKEN.").Envar("GITHUB_TOKEN").String()
githubIncludeForks = githubScan.Flag("include-forks", "Include forks in scan.").Bool()
githubIncludeMembers = githubScan.Flag("include-members", "Include organization member repositories in scan.").Bool()
githubIncludeRepos = githubScan.Flag("include-repos", `Repositories to include in an org scan. This can also be a glob pattern. You can repeat this flag. Must use Github repo full name. Example: "trufflesecurity/trufflehog", "trufflesecurity/t*"`).Strings()
githubExcludeRepos = githubScan.Flag("exclude-repos", `Repositories to exclude in an org scan. This can also be a glob pattern. You can repeat this flag. Must use Github repo full name. Example: "trufflesecurity/driftwood", "trufflesecurity/d*"`).Strings()
gitlabScan = cli.Command("gitlab", "Find credentials in GitLab repositories.")
// TODO: Add more GitLab options
@@ -87,14 +76,12 @@ var (
gitlabScanIncludePaths = gitlabScan.Flag("include-paths", "Path to file with newline separated regexes for files to include in scan.").Short('i').String()
gitlabScanExcludePaths = gitlabScan.Flag("exclude-paths", "Path to file with newline separated regexes for files to exclude in scan.").Short('x').String()
filesystemScan = cli.Command("filesystem", "Find credentials in a filesystem.")
filesystemPaths = filesystemScan.Arg("path", "Path to file or directory to scan.").Strings()
// DEPRECATED: --directory is deprecated in favor of arguments.
filesystemDirectories = filesystemScan.Flag("directory", "Path to directory to scan. You can repeat this flag.").Strings()
filesystemScan = cli.Command("filesystem", "Find credentials in a filesystem.")
filesystemDirectories = filesystemScan.Flag("directory", "Path to directory to scan. You can repeat this flag.").Required().Strings()
// TODO: Add more filesystem scan options. Currently only supports scanning a list of directories.
// filesystemScanRecursive = filesystemScan.Flag("recursive", "Scan recursively.").Short('r').Bool()
filesystemScanIncludePaths = filesystemScan.Flag("include-paths", "Path to file with newline separated regexes for files to include in scan.").Short('i').String()
filesystemScanExcludePaths = filesystemScan.Flag("exclude-paths", "Path to file with newline separated regexes for files to exclude in scan.").Short('x').String()
// filesystemScanIncludePaths = filesystemScan.Flag("include-paths", "Path to file with newline separated regexes for files to include in scan.").Short('i').String()
// filesystemScanExcludePaths = filesystemScan.Flag("exclude-paths", "Path to file with newline separated regexes for files to exclude in scan.").Short('x').String()
s3Scan = cli.Command("s3", "Find credentials in S3 buckets.")
s3ScanKey = s3Scan.Flag("key", "S3 key used to authenticate. Can be provided with environment variable AWS_ACCESS_KEY_ID.").Envar("AWS_ACCESS_KEY_ID").String()
@@ -125,26 +112,24 @@ func init() {
cli.Version("trufflehog " + version.BuildVersion)
cmd = kingpin.MustParse(cli.Parse(os.Args[1:]))
if *jsonOut {
logrus.SetFormatter(&logrus.JSONFormatter{})
}
switch {
case *trace:
log.SetLevel(5)
logrus.SetLevel(logrus.TraceLevel)
logrus.Debugf("running version %s", version.BuildVersion)
case *debug:
log.SetLevel(2)
logrus.SetLevel(logrus.DebugLevel)
logrus.Debugf("running version %s", version.BuildVersion)
default:
logrus.SetLevel(logrus.InfoLevel)
}
}
func main() {
// setup logger
logFormat := log.WithConsoleSink
if *jsonOut {
logFormat = log.WithJSONSink
}
logger, sync := log.New("trufflehog", logFormat(os.Stderr))
// make it the default logger for contexts
context.SetDefaultLogger(logger)
defer func() { _ = sync() }()
logFatal := logFatalFunc(logger)
updateCfg := overseer.Config{
Program: run,
Debug: *debug,
@@ -162,16 +147,14 @@ func main() {
err := overseer.RunErr(updateCfg)
if err != nil {
logFatal(err, "error occured with trufflehog updater 🐷")
logrus.WithError(err).Fatal("error occured with trufflehog updater 🐷")
}
}
func run(state overseer.State) {
ctx := context.Background()
logger := ctx.Logger()
logFatal := logFatalFunc(logger)
logger.V(2).Info(fmt.Sprintf("trufflehog %s", version.BuildVersion))
if *debug {
logrus.Debugf("trufflehog %s", version.BuildVersion)
}
if *githubScanToken != "" {
// NOTE: this kludge is here to do an authenticated shallow commit
@@ -184,182 +167,138 @@ func run(state overseer.State) {
*concurrency = 1
}
if *profile {
if *debug {
go func() {
router := mux.NewRouter()
router.PathPrefix("/debug/pprof").Handler(http.DefaultServeMux)
router.PathPrefix("/debug/fgprof").Handler(fgprof.Handler())
logger.Info("starting pprof and fgprof server on :18066 /debug/pprof and /debug/fgprof")
logrus.Info("starting pprof and fgprof server on :18066 /debug/pprof and /debug/fgprof")
if err := http.ListenAndServe(":18066", router); err != nil {
logger.Error(err, "error serving pprof and fgprof")
logrus.Error(err)
}
}()
}
logger, sync := log.New("trufflehog", log.WithConsoleSink(os.Stderr))
context.SetDefaultLogger(logger)
defer func() { _ = sync() }()
conf := &config.Config{}
if *configFilename != "" {
var err error
conf, err = config.Read(*configFilename)
if err != nil {
logFatal(err, "error parsing the provided configuration file")
}
}
if *archiveMaxSize != 0 {
handlers.SetArchiveMaxSize(int(*archiveMaxSize))
}
if *archiveMaxDepth != 0 {
handlers.SetArchiveMaxDepth(*archiveMaxDepth)
}
if *archiveTimeout != 0 {
handlers.SetArchiveMaxTimeout(*archiveTimeout)
}
// Build include and exclude detector filter sets.
var includeDetectorTypes, excludeDetectorTypes map[detectorspb.DetectorType]struct{}
{
includeList, err := config.ParseDetectors(*includeDetectors)
if err != nil {
// Exit if there was an error to inform the user of the misconfiguration.
logger.Error(err, "invalid include list detector configuration")
logger.Error(err, "error parsing the provided configuration file")
os.Exit(1)
}
excludeList, err := config.ParseDetectors(*excludeDetectors)
if err != nil {
// Exit if there was an error to inform the user of the misconfiguration.
logger.Error(err, "invalid exclude list detector configuration")
os.Exit(1)
}
includeDetectorTypes = detectorTypeToSet(includeList)
excludeDetectorTypes = detectorTypeToSet(excludeList)
}
includeFilter := func(d detectors.Detector) bool {
_, ok := includeDetectorTypes[d.Type()]
return ok
}
excludeFilter := func(d detectors.Detector) bool {
_, ok := excludeDetectorTypes[d.Type()]
return !ok
}
ctx := context.TODO()
e := engine.Start(ctx,
engine.WithConcurrency(*concurrency),
engine.WithDecoders(decoders.DefaultDecoders()...),
engine.WithDetectors(!*noVerification, engine.DefaultDetectors()...),
engine.WithDetectors(!*noVerification, conf.Detectors...),
engine.WithFilterDetectors(includeFilter),
engine.WithFilterDetectors(excludeFilter),
engine.WithFilterUnverified(*filterUnverified),
)
filter, err := common.FilterFromFiles(*gitScanIncludePaths, *gitScanExcludePaths)
if err != nil {
logrus.WithError(err).Fatal("could not create filter")
}
var repoPath string
var remote bool
switch cmd {
case gitScan.FullCommand():
filter, err := common.FilterFromFiles(*gitScanIncludePaths, *gitScanExcludePaths)
if err != nil {
logFatal(err, "could not create filter")
}
repoPath, remote, err = git.PrepareRepoSinceCommit(ctx, *gitScanURI, *gitScanSinceCommit)
if err != nil || repoPath == "" {
logFatal(err, "error preparing git repo for scanning")
logrus.WithError(err).Fatal("error preparing git repo for scanning")
}
if remote {
defer os.RemoveAll(repoPath)
}
cfg := sources.GitConfig{
RepoPath: repoPath,
HeadRef: *gitScanBranch,
BaseRef: *gitScanSinceCommit,
MaxDepth: *gitScanMaxDepth,
Filter: filter,
}
if err = e.ScanGit(ctx, cfg); err != nil {
logFatal(err, "Failed to scan Git.")
}
case githubScan.FullCommand():
filter, err := common.FilterFromFiles(*githubScanIncludePaths, *githubScanExcludePaths)
if err != nil {
logFatal(err, "could not create filter")
}
if len(*githubScanOrgs) == 0 && len(*githubScanRepos) == 0 {
logFatal(fmt.Errorf("invalid config"), "You must specify at least one organization or repository.")
g := func(c *sources.Config) {
c.RepoPath = repoPath
c.HeadRef = *gitScanBranch
c.BaseRef = *gitScanSinceCommit
c.MaxDepth = *gitScanMaxDepth
c.Filter = filter
}
cfg := sources.GithubConfig{
Endpoint: *githubScanEndpoint,
Token: *githubScanToken,
IncludeForks: *githubIncludeForks,
IncludeMembers: *githubIncludeMembers,
Concurrency: *concurrency,
ExcludeRepos: *githubExcludeRepos,
IncludeRepos: *githubIncludeRepos,
Repos: *githubScanRepos,
Orgs: *githubScanOrgs,
Filter: filter,
if err = e.ScanGit(ctx, sources.NewConfig(g)); err != nil {
logrus.WithError(err).Fatal("Failed to scan Git.")
}
if err := e.ScanGitHub(ctx, cfg); err != nil {
logFatal(err, "Failed to scan Github.")
case githubScan.FullCommand():
if len(*githubScanOrgs) == 0 && len(*githubScanRepos) == 0 {
logrus.Fatal("You must specify at least one organization or repository.")
}
github := func(c *sources.Config) {
c.Endpoint = *githubScanEndpoint
c.Repos = *githubScanRepos
c.Orgs = *githubScanOrgs
c.Token = *githubScanToken
c.IncludeForks = *githubIncludeForks
c.IncludeMembers = *githubIncludeMembers
c.Concurrency = *concurrency
c.ExcludeRepos = *githubExcludeRepos
c.IncludeRepos = *githubIncludeRepos
}
if err = e.ScanGitHub(ctx, sources.NewConfig(github)); err != nil {
logrus.WithError(err).Fatal("Failed to scan Github.")
}
case gitlabScan.FullCommand():
filter, err := common.FilterFromFiles(*gitlabScanIncludePaths, *gitlabScanExcludePaths)
if err != nil {
logFatal(err, "could not create filter")
logrus.WithError(err).Fatal("could not create filter")
}
cfg := sources.GitlabConfig{
Endpoint: *gitlabScanEndpoint,
Token: *gitlabScanToken,
Repos: *gitlabScanRepos,
Filter: filter,
gitlab := func(c *sources.Config) {
c.Endpoint = *gitlabScanEndpoint
c.Token = *gitlabScanToken
c.Repos = *gitlabScanRepos
c.Filter = filter
}
if err := e.ScanGitLab(ctx, cfg); err != nil {
logFatal(err, "Failed to scan GitLab.")
if err = e.ScanGitLab(ctx, sources.NewConfig(gitlab)); err != nil {
logrus.WithError(err).Fatal("Failed to scan GitLab.")
}
case filesystemScan.FullCommand():
filter, err := common.FilterFromFiles(*filesystemScanIncludePaths, *filesystemScanExcludePaths)
if err != nil {
logFatal(err, "could not create filter")
fs := func(c *sources.Config) {
c.Directories = *filesystemDirectories
}
if len(*filesystemDirectories) > 0 {
ctx.Logger().Info("--directory flag is deprecated, please pass directories as arguments")
}
paths := make([]string, 0, len(*filesystemPaths)+len(*filesystemDirectories))
paths = append(paths, *filesystemPaths...)
paths = append(paths, *filesystemDirectories...)
cfg := sources.FilesystemConfig{
Paths: paths,
Filter: filter,
}
if err = e.ScanFileSystem(ctx, cfg); err != nil {
logFatal(err, "Failed to scan filesystem")
if err = e.ScanFileSystem(ctx, sources.NewConfig(fs)); err != nil {
logrus.WithError(err).Fatal("Failed to scan filesystem")
}
case s3Scan.FullCommand():
cfg := sources.S3Config{
Key: *s3ScanKey,
Secret: *s3ScanSecret,
Buckets: *s3ScanBuckets,
CloudCred: *s3ScanCloudEnv,
s3 := func(c *sources.Config) {
c.Key = *s3ScanKey
c.Secret = *s3ScanSecret
c.Buckets = *s3ScanBuckets
}
if err := e.ScanS3(ctx, cfg); err != nil {
logFatal(err, "Failed to scan S3.")
if err = e.ScanS3(ctx, sources.NewConfig(s3)); err != nil {
logrus.WithError(err).Fatal("Failed to scan S3.")
}
case syslogScan.FullCommand():
cfg := sources.SyslogConfig{
Address: *syslogAddress,
Format: *syslogFormat,
Protocol: *syslogProtocol,
CertPath: *syslogTLSCert,
KeyPath: *syslogTLSKey,
Concurrency: *concurrency,
syslog := func(c *sources.Config) {
c.Address = *syslogAddress
c.Protocol = *syslogProtocol
c.CertPath = *syslogTLSCert
c.KeyPath = *syslogTLSKey
c.Format = *syslogFormat
c.Concurrency = *concurrency
}
if err := e.ScanSyslog(ctx, cfg); err != nil {
logFatal(err, "Failed to scan syslog.")
if err = e.ScanSyslog(ctx, sources.NewConfig(syslog)); err != nil {
logrus.WithError(err).Fatal("Failed to scan syslog.")
}
case circleCiScan.FullCommand():
if err := e.ScanCircleCI(ctx, *circleCiScanToken); err != nil {
logFatal(err, "Failed to scan CircleCI.")
if err = e.ScanCircleCI(ctx, *circleCiScanToken); err != nil {
logrus.WithError(err).Fatal("Failed to scan CircleCI.")
}
}
// asynchronously wait for scanning to finish and cleanup
@@ -378,30 +317,24 @@ func run(state overseer.State) {
}
foundResults = true
var err error
switch {
case *jsonLegacy:
err = output.PrintLegacyJSON(ctx, &r)
output.PrintLegacyJSON(ctx, &r)
case *jsonOut:
err = output.PrintJSON(&r)
output.PrintJSON(&r)
default:
err = output.PrintPlainOutput(&r)
}
if err != nil {
logFatal(err, "error printing results")
output.PrintPlainOutput(&r)
}
}
logger.V(2).Info("finished scanning",
"chunks", e.ChunksScanned(),
"bytes", e.BytesScanned(),
)
logrus.Debugf("scanned %d chunks", e.ChunksScanned())
logrus.Debugf("scanned %d bytes", e.BytesScanned())
if *printAvgDetectorTime {
printAverageDetectorTime(e)
}
if foundResults && *fail {
logger.V(2).Info("exiting with code 183 because results were found")
logrus.Debug("exiting with code 183 because results were found")
os.Exit(183)
}
}
@@ -417,24 +350,3 @@ func printAverageDetectorTime(e *engine.Engine) {
fmt.Fprintf(os.Stderr, "%s: %s\n", detectorName, avgDuration)
}
}
// logFatalFunc returns a log.Fatal style function. Calling the returned
// function will terminate the program without cleanup.
func logFatalFunc(logger logr.Logger) func(error, string, ...any) {
return func(err error, message string, keyAndVals ...any) {
logger.Error(err, message, keyAndVals...)
if err != nil {
os.Exit(1)
return
}
os.Exit(0)
}
}
func detectorTypeToSet(detectors []detectorspb.DetectorType) map[detectorspb.DetectorType]struct{} {
output := make(map[detectorspb.DetectorType]struct{}, len(detectors))
for _, d := range detectors {
output[d] = struct{}{}
}
return output
}
+4 -12
View File
@@ -6,7 +6,7 @@ import (
"os"
"regexp"
"github.com/trufflesecurity/trufflehog/v3/pkg/context"
log "github.com/sirupsen/logrus"
)
type Filter struct {
@@ -20,8 +20,7 @@ type FilterRuleSet []regexp.Regexp
func FilterEmpty() *Filter {
filter, err := FilterFromFiles("", "")
if err != nil {
context.Background().Logger().Error(err, "could not create empty filter")
os.Exit(1)
log.WithError(err).Fatalf("could not create empty filter")
}
return filter
}
@@ -58,19 +57,15 @@ func FilterRulesFromFile(source string) (*FilterRuleSet, error) {
}
commentPattern := regexp.MustCompile(`^\s*#`)
emptyLinePattern := regexp.MustCompile(`^\s*$`)
file, err := os.Open(source)
logger := context.Background().Logger().WithValues("file", source)
if err != nil {
logger.Error(err, "unable to open filter file", "file", source)
os.Exit(1)
log.WithError(err).Fatalf("unable to open filter file: %s", source)
}
defer func(file *os.File) {
err := file.Close()
if err != nil {
logger.Error(err, "unable to close filter file")
os.Exit(1)
log.WithError(err).Fatalf("unable to close filter file: %s", source)
}
}(file)
@@ -80,9 +75,6 @@ func FilterRulesFromFile(source string) (*FilterRuleSet, error) {
if commentPattern.MatchString(line) {
continue
}
if emptyLinePattern.MatchString(line) {
continue
}
pattern, err := regexp.Compile(line)
if err != nil {
return nil, fmt.Errorf("can not compile regular expression: %s", line)
-7
View File
@@ -129,13 +129,6 @@ func TestFilterFromFile(t *testing.T) {
pattern: "test",
pass: false,
},
"EmptyLinesAreIgnored": {
includeFile: false,
excludeFile: true,
excludeFileContents: " \ntest.txt",
pattern: "hello world.txt",
pass: true,
},
}
for name, test := range tests {
var includeTestFile, excludeTestFile string
+4 -3
View File
@@ -9,6 +9,7 @@ import (
"time"
"github.com/hashicorp/go-retryablehttp"
"github.com/sirupsen/logrus"
)
var caCerts = []string{
@@ -93,7 +94,7 @@ func NewCustomTransport(T http.RoundTripper) *CustomTransport {
func PinnedRetryableHttpClient() *http.Client {
httpClient := retryablehttp.NewClient()
httpClient.Logger = nil
httpClient.Logger = logrus.WithField("name", "PinnedRetryableHttpClient").WithField("level", logrus.DebugLevel)
httpClient.HTTPClient.Transport = NewCustomTransport(&http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: PinnedCertPool(),
@@ -115,7 +116,7 @@ func PinnedRetryableHttpClient() *http.Client {
func RetryableHttpClient() *http.Client {
httpClient := retryablehttp.NewClient()
httpClient.RetryMax = 3
httpClient.Logger = nil
httpClient.Logger = logrus.WithField("name", "RetryableHttpClient").WithField("level", logrus.DebugLevel)
httpClient.HTTPClient.Timeout = 3 * time.Second
httpClient.HTTPClient.Transport = NewCustomTransport(nil)
return httpClient.StandardClient()
@@ -124,7 +125,7 @@ func RetryableHttpClient() *http.Client {
func RetryableHttpClientTimeout(timeOutSeconds int64) *http.Client {
httpClient := retryablehttp.NewClient()
httpClient.RetryMax = 3
httpClient.Logger = nil
httpClient.Logger = logrus.WithField("name", "RetryableHttpClientTimeout").WithField("level", logrus.DebugLevel)
httpClient.HTTPClient.Timeout = time.Duration(timeOutSeconds) * time.Second
httpClient.HTTPClient.Transport = NewCustomTransport(nil)
return httpClient.StandardClient()
+3 -1
View File
@@ -4,6 +4,8 @@ import (
"fmt"
"strconv"
"strings"
log "github.com/sirupsen/logrus"
)
const EmailPattern = `\b(?:[a-z0-9!#$%&'*+/=?^_\x60{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_\x60{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])\b`
@@ -22,7 +24,7 @@ func BuildRegex(pattern string, specialChar string, length int) string {
func BuildRegexJWT(firstRange, secondRange, thirdRange string) string {
if RangeValidation(firstRange) || RangeValidation(secondRange) || RangeValidation(thirdRange) {
panic("Min value should not be greater than or equal to max")
log.Error("Min value should not be greater than or equal to max")
}
return fmt.Sprintf(`\b(ey[%s]{%s}.ey[%s-\/_]{%s}.[%s-\/_]{%s})\b`, AlphaNumPattern, firstRange, AlphaNumPattern, secondRange, AlphaNumPattern, thirdRange)
}
+1 -1
View File
@@ -7,9 +7,9 @@ import (
"time"
secretmanager "cloud.google.com/go/secretmanager/apiv1"
"cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
"github.com/joho/godotenv"
"github.com/pkg/errors"
secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"
)
type Secret struct{ kv map[string]string }
-140
View File
@@ -1,140 +0,0 @@
package config
import (
"fmt"
"strconv"
"strings"
dpb "github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
"golang.org/x/exp/slices"
)
var (
specialGroups = map[string][]dpb.DetectorType{
"all": allDetectors(),
}
detectorTypeValue = make(map[string]dpb.DetectorType, len(dpb.DetectorType_value))
validDetectors = make(map[dpb.DetectorType]struct{}, len(dpb.DetectorType_value))
maxDetectorType dpb.DetectorType
)
// Setup package local global variables.
func init() {
for k, v := range dpb.DetectorType_value {
dt := dpb.DetectorType(v)
detectorTypeValue[strings.ToLower(k)] = dt
validDetectors[dt] = struct{}{}
if dt > maxDetectorType {
maxDetectorType = dt
}
}
}
// ParseDetectors parses user supplied string into a list of detectors types.
// "all" will return the list of all available detectors. The input is comma
// separated and may use the case-insensitive detector name defined in the
// protobuf, or the protobuf enum number. A range may be used as well in the
// form "start-end". Order is preserved and duplicates are ignored.
func ParseDetectors(input string) ([]dpb.DetectorType, error) {
var output []dpb.DetectorType
seenDetector := map[dpb.DetectorType]struct{}{}
for _, item := range strings.Split(input, ",") {
item = strings.TrimSpace(item)
if item == "" {
continue
}
detectors, ok := specialGroups[strings.ToLower(item)]
if !ok {
var err error
detectors, err = asRange(item)
if err != nil {
return nil, err
}
}
for _, d := range detectors {
if _, ok := seenDetector[d]; ok {
continue
}
seenDetector[d] = struct{}{}
output = append(output, d)
}
}
return output, nil
}
// allDetectors reutrns an ordered slice of all detector types.
func allDetectors() []dpb.DetectorType {
all := make([]dpb.DetectorType, 0, len(dpb.DetectorType_name))
for id := range dpb.DetectorType_name {
all = append(all, dpb.DetectorType(id))
}
slices.Sort(all)
return all
}
// asRange converts a single input into a slice of detector types. If the input
// is not in range format, a slice of length 1 is returned. Unbounded ranges
// are allowed.
func asRange(input string) ([]dpb.DetectorType, error) {
// Check if it's a single detector type.
dt, err := asDetectorType(input)
if err == nil {
return []dpb.DetectorType{dt}, nil
}
// Check if it's a range; if not return the error from above.
start, end, found := strings.Cut(input, "-")
if !found {
return nil, err
}
start, end = strings.TrimSpace(start), strings.TrimSpace(end)
// Convert the range start and end to a DetectorType.
dtStart, err := asDetectorType(start)
if err != nil {
return nil, err
}
dtEnd, err := asDetectorType(end)
// If end is empty it's an unbounded range.
if err != nil && end != "" {
return nil, err
}
if end == "" {
dtEnd = maxDetectorType
}
step := dpb.DetectorType(1)
if dtStart > dtEnd {
step = -1
}
var output []dpb.DetectorType
for dt := dtStart; dt != dtEnd; dt += step {
if _, ok := validDetectors[dt]; !ok {
continue
}
output = append(output, dt)
}
return append(output, dtEnd), nil
}
// asDetectorType converts the case-insensitive input into a detector type.
// Name or ID may be used.
func asDetectorType(input string) (dpb.DetectorType, error) {
if input == "" {
return 0, fmt.Errorf("empty detector")
}
// Check if it's a named detector.
if dt, ok := detectorTypeValue[strings.ToLower(input)]; ok {
return dt, nil
}
// Check if it's a detector ID.
if i, err := strconv.ParseInt(input, 10, 32); err == nil {
dt := dpb.DetectorType(i)
if _, ok := validDetectors[dt]; !ok {
return 0, fmt.Errorf("invalid detector ID: %s", input)
}
return dt, nil
}
return 0, fmt.Errorf("unrecognized detector type: %s", input)
}
-40
View File
@@ -1,40 +0,0 @@
package config
import (
"testing"
"github.com/stretchr/testify/assert"
dpb "github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
)
func TestDetectorParsing(t *testing.T) {
tests := map[string]struct {
input string
expected []dpb.DetectorType
}{
"all": {"AlL", allDetectors()},
"trailing range": {"0-", allDetectors()},
"all after 1": {"1-", allDetectors()[1:]},
"named and valid range": {"aWs,8-9", []dpb.DetectorType{dpb.DetectorType_AWS, dpb.DetectorType_Github, dpb.DetectorType_Gitlab}},
"duplicate order preserved": {"9, 8, 9", []dpb.DetectorType{9, 8}},
"named range": {"github - gitlab", []dpb.DetectorType{dpb.DetectorType_Github, dpb.DetectorType_Gitlab}},
"range preserved": {"8-9, 7-10", []dpb.DetectorType{8, 9, 7, 10}},
"reverse range": {"9-8", []dpb.DetectorType{9, 8}},
"range preserved with all": {"10-,all", append(allDetectors()[10:], allDetectors()[:10]...)},
"empty list item": {"8, ,9", []dpb.DetectorType{8, 9}},
"invalid end range": {"0-1337", nil},
"invalid name": {"foo", nil},
"negative": {"-1", nil},
}
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
got, gotErr := ParseDetectors(tt.input)
if tt.expected == nil {
assert.Error(t, gotErr)
return
}
assert.Equal(t, tt.expected, got)
})
}
}
+1 -4
View File
@@ -166,9 +166,6 @@ func captureCancelCallstack(ctx logCtx, f context.CancelFunc) (Context, context.
}
f()
// Set the error with the stacktrace if the err pointer is non-nil.
*ctx.err = fmt.Errorf(
"%w (canceled at %v\n%s)",
ctx.Err(), time.Now(), string(debug.Stack()),
)
*ctx.err = fmt.Errorf("%w (%s)", ctx.Err(), string(debug.Stack()))
}
}
+67 -87
View File
@@ -12,7 +12,6 @@ import (
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/custom_detectorspb"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
"golang.org/x/sync/errgroup"
)
// The maximum number of matches from one chunk. This const is used when
@@ -83,99 +82,71 @@ func (c *customRegexWebhook) FromData(ctx context.Context, verify bool, data []b
// ]
matches := permutateMatches(regexMatches)
g := new(errgroup.Group)
// Create result object and test for verification.
resultsCh := make(chan detectors.Result, maxTotalMatches)
for _, match := range matches {
match := match
g.Go(func() error {
return c.createResults(ctx, match, verify, resultsCh)
if common.IsDone(ctx) {
// TODO: Log we're possibly leaving out results.
return results, nil
}
var raw string
for _, values := range match {
// values[0] contains the entire regex match.
raw += values[0]
}
result := detectors.Result{
DetectorType: detectorspb.DetectorType_CustomRegex,
Raw: []byte(raw),
}
if isKnownFalsePositive(match) {
continue
}
if !verify {
results = append(results, result)
continue
}
// Verify via webhook.
jsonBody, err := json.Marshal(map[string]map[string][]string{
c.GetName(): match,
})
}
if err := g.Wait(); err != nil {
return nil, err
}
close(resultsCh)
for result := range resultsCh {
if err != nil {
continue
}
// Try each config until we successfully verify.
for _, verifyConfig := range c.GetVerify() {
if common.IsDone(ctx) {
// TODO: Log we're possibly leaving out results.
return results, nil
}
req, err := http.NewRequestWithContext(ctx, "POST", verifyConfig.GetEndpoint(), bytes.NewReader(jsonBody))
if err != nil {
continue
}
for _, header := range verifyConfig.GetHeaders() {
key, value, found := strings.Cut(header, ":")
if !found {
// Should be unreachable due to validation.
continue
}
req.Header.Add(key, strings.TrimLeft(value, "\t\n\v\f\r "))
}
res, err := httpClient.Do(req)
if err != nil {
continue
}
// TODO: Read response body.
res.Body.Close()
if res.StatusCode == http.StatusOK {
result.Verified = true
break
}
}
results = append(results, result)
}
return results, nil
}
func (c *customRegexWebhook) createResults(ctx context.Context, match map[string][]string, verify bool, results chan<- detectors.Result) error {
if common.IsDone(ctx) {
// TODO: Log we're possibly leaving out results.
return ctx.Err()
}
var raw string
for _, values := range match {
// values[0] contains the entire regex match.
raw += values[0]
}
result := detectors.Result{
DetectorType: detectorspb.DetectorType_CustomRegex,
Raw: []byte(raw),
}
if !verify {
select {
case <-ctx.Done():
return ctx.Err()
case results <- result:
return nil
}
}
// Verify via webhook.
jsonBody, err := json.Marshal(map[string]map[string][]string{
c.GetName(): match,
})
if err != nil {
// This should never happen, but if it does, return nil to not
// disrupt other verification.
return nil
}
// Try each config until we successfully verify.
for _, verifyConfig := range c.GetVerify() {
if common.IsDone(ctx) {
// TODO: Log we're possibly leaving out results.
return ctx.Err()
}
req, err := http.NewRequestWithContext(ctx, "POST", verifyConfig.GetEndpoint(), bytes.NewReader(jsonBody))
if err != nil {
continue
}
for _, header := range verifyConfig.GetHeaders() {
key, value, found := strings.Cut(header, ":")
if !found {
// Should be unreachable due to validation.
continue
}
req.Header.Add(key, strings.TrimLeft(value, "\t\n\v\f\r "))
}
res, err := httpClient.Do(req)
if err != nil {
continue
}
// TODO: Read response body.
res.Body.Close()
if res.StatusCode == http.StatusOK {
result.Verified = true
break
}
}
select {
case <-ctx.Done():
return ctx.Err()
case results <- result:
return nil
}
}
func (c *customRegexWebhook) Keywords() []string {
return c.GetKeywords()
}
@@ -241,6 +212,15 @@ func permutateMatches(regexMatches map[string][][]string) []map[string][]string
return matches
}
func (c *customRegexWebhook) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_CustomRegex
// This function will check false positives for common test words, but also it
// will make sure the key appears 'random' enough to be a real key.
func isKnownFalsePositive(match map[string][]string) bool {
for _, values := range match {
for _, value := range values {
if detectors.IsKnownFalsePositive(value, detectors.DefaultFalsePositives, true) {
return true
}
}
}
return false
}
@@ -193,21 +193,6 @@ func TestPermutateMatches(t *testing.T) {
}
}
func TestDetector(t *testing.T) {
detector, err := NewWebhookCustomRegex(&custom_detectorspb.CustomRegex{
Name: "test",
// "password" is normally flagged as a false positive, but CustomRegex
// should allow the user to decide and report it as a result.
Keywords: []string{"password"},
Regex: map[string]string{"regex": "password=.*"},
})
assert.NoError(t, err)
results, err := detector.FromData(context.Background(), false, []byte(`password="123456"`))
assert.NoError(t, err)
assert.Equal(t, 1, len(results))
assert.Equal(t, results[0].Raw, []byte(`password="123456"`))
}
func BenchmarkProductIndices(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = productIndices(3, 2, 6)
+2 -5
View File
@@ -2,6 +2,7 @@ package abbysale
import (
"context"
"fmt"
"net/http"
"regexp"
"strings"
@@ -51,7 +52,7 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
if err != nil {
continue
}
req.Header.Add("x-api-key", resMatch)
req.Header.Add("x-api-key", fmt.Sprintf("%s", resMatch))
res, err := client.Do(req)
if err == nil {
defer res.Body.Close()
@@ -71,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Abbysale
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Abstract
}
-4
View File
@@ -83,7 +83,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AbuseIPDB
}
-4
View File
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Accuweather
}
-4
View File
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AdafruitIO
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AdobeIO
}
-4
View File
@@ -80,7 +80,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Adzuna
}
@@ -83,7 +83,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Aeroworkflow
}
-4
View File
@@ -83,7 +83,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Agora
}
-4
View File
@@ -73,7 +73,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Aha
}
@@ -87,7 +87,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AirbrakeProjectKey
}
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AirbrakeUserKey
}
-4
View File
@@ -73,7 +73,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Airship
}
@@ -81,7 +81,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AirtableApiKey
}
-4
View File
@@ -73,7 +73,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AirVisual
}
-4
View File
@@ -76,7 +76,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Alconost
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Alegra
}
-4
View File
@@ -74,7 +74,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AletheiaApi
}
@@ -80,7 +80,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
}
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AlgoliaAdminKey
}
-4
View File
@@ -130,7 +130,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Alibaba
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AlienVault
}
-4
View File
@@ -77,7 +77,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Allsports
}
-4
View File
@@ -88,7 +88,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Amadeus
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Ambee
}
@@ -81,7 +81,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AmplitudeApiKey
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Anypoint
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apacta
}
-4
View File
@@ -86,7 +86,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
type Response struct {
ReturnCode int `json:"return_code"`
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Api2Cart
}
-4
View File
@@ -84,7 +84,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_ApiDeck
}
-4
View File
@@ -80,7 +80,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apiflash
}
-4
View File
@@ -83,7 +83,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_ApiFonica
}
-4
View File
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apify
}
-4
View File
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apilayer
}
-4
View File
@@ -84,7 +84,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
}
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_APIMatic
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_ApiScience
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_APITemplate
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apollo
}
-4
View File
@@ -92,7 +92,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Appcues
}
-4
View File
@@ -75,7 +75,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Appfollow
}
-4
View File
@@ -76,7 +76,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Appointedd
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AppSynergy
}
-4
View File
@@ -91,7 +91,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Apptivo
}
-4
View File
@@ -83,7 +83,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_ArtifactoryAccessToken
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Artsy
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AsanaOauth
}
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AsanaPersonalAccessToken
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AssemblyAI
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Atera
}
-4
View File
@@ -81,7 +81,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Audd
}
@@ -90,7 +90,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Auth0ManagementApiToken
}
-4
View File
@@ -116,7 +116,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Auth0oauth
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Autodesk
}
-4
View File
@@ -81,7 +81,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Autoklose
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AutoPilot
}
@@ -73,7 +73,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AvazaPersonalAccessToken
}
@@ -74,7 +74,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AviationStack
}
-4
View File
@@ -249,7 +249,3 @@ type identityRes struct {
} `json:"ResponseMetadata"`
} `json:"GetCallerIdentityResponse"`
}
func (s scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_AWS
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Axonaut
}
-4
View File
@@ -81,7 +81,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Aylien
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Ayrshare
}
-4
View File
@@ -85,7 +85,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Azure
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Bannerbear
}
-4
View File
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Baremetrics
}
-4
View File
@@ -79,7 +79,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_BaseApiIO
}
-4
View File
@@ -71,7 +71,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Beamer
}
-4
View File
@@ -77,7 +77,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Beebole
}
-4
View File
@@ -74,7 +74,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Besnappy
}
-4
View File
@@ -77,7 +77,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Besttime
}
-4
View File
@@ -82,7 +82,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Billomat
}
-4
View File
@@ -76,7 +76,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Bitbar
}
@@ -70,7 +70,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_BitcoinAverage
}
+1 -5
View File
@@ -86,7 +86,7 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
s1.Verified = isValid
// If there is a valid one, we need to stop iterating now and return the valid result
if isValid {
if isValid == true {
break
}
}
@@ -101,7 +101,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Bitfinex
}
@@ -72,7 +72,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_BitLyAccessToken
}
-4
View File
@@ -109,7 +109,3 @@ func getBitmexSignature(timeStamp string, secret string, action string, path str
macsum := mac.Sum(nil)
return hex.EncodeToString(macsum)
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Bitmex
}
-4
View File
@@ -73,7 +73,3 @@ func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (result
return results, nil
}
func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_Blablabus
}

Some files were not shown because too many files have changed in this diff Show More