Commit Graph

1814 Commits

Author SHA1 Message Date
Miccah Castorina dc4b2bd5e2 Use SourceManager in Engine
And subsequently all of the sources in the engine package.
2023-07-23 08:36:57 -05:00
Miccah Castorina 67860315e7 Miscellaneous SourceManager updates
* Own the chunks channel instead of accepting it as an input
* Add Chunks and Wait methods
* Fix bug in Enroll so it actually returns the handle
* Add context.Context parameter to the SourceInitFunc type
2023-07-23 08:36:57 -05:00
Miccah Castorina f4ce4b4ac9 Preserve ScanGit functionality of not cleaning up temporary files 2023-07-23 08:36:19 -05:00
Miccah Castorina dfd1845400 Refactor git source to allow ScanOptions and use source in engine
Refactor the Chunks method of the git Source to call out to two helper
methods: scanRepos and scanDirs which scans s.conn.Repositories and
s.conn.Directories respectively. The only notable change in behavior is
that a credential is no longer necessary if there are no
s.conn.Repositories to scan.
2023-07-23 08:36:19 -05:00
ahrav 9e0a2e9ddd [chore] - Remove password info from log (#1528)
* Remove password info from log.

* update.

* one more.
2023-07-22 20:25:45 -07:00
Miccah 91c5472876 Implement SourceManager basics (#1515)
* Implement SourceManager basics

* Rename identifiers and add a default headlessAPI implementation

* Rewrite to use SourceInitFunc

* Update variable name to accurately reflect its value
2023-07-21 15:20:25 -05:00
Miccah a613bbb979 [chore] Remove parent manipulation in context package (#1525)
The ability to set the parent allowed creating context cycles which
shouldn't be allowed, or at the very least have unintuitive behavior.
2023-07-21 13:51:51 -05:00
Cody Rose ebf1038392 Support indeterminacy in alchemy and update detector docs (#1510) 2023-07-21 14:50:14 -04:00
Cody Rose 06a562688d capture json error (#1509) 2023-07-21 10:44:47 -05:00
Miccah e8b5e3cea3 Revert "[chore] Remove parent setting / getting in Context wrapper (#1516)" (#1519)
This reverts commit 8ec5e4916c.
This commit is somehow causing AWS verification (and possibly others) to
not work.
2023-07-20 23:31:28 -05:00
Miccah 8ec5e4916c [chore] Remove parent setting / getting in Context wrapper (#1516)
* [chore] Remove parent setting / getting in Context wrapper

* Keep the cancellable context from errgroup
2023-07-20 13:33:09 -05:00
Cody Rose 20b7793828 JDBC indeterminacy (#1507)
This PR adds an indeterminacy check to the JDBC verifiers.
2023-07-19 16:57:57 -04:00
Brandon Yan 8fad5fff79 add dockerhub scanner (#1496)
* add dockerhub scanner

* clean

* clean and fix regex logic and tests

* check length of userMatches before access

* Use camelcase.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-19 09:26:28 -07:00
Cody Rose cb1a63a4e2 unify JDBC detector ping logic (#1506)
Previously, the various JDBC detectors would independently try to verify credentials by a process of trying various permutations of candidates one-by-one. The upcoming tri-state verification work will need to add sophistication to this process in the same way for each one, so this PR first combines all of the logic so it can be upgraded in a single spot.
2023-07-19 11:45:56 -04:00
Zubair Khan be549a7287 add thog enterprise detector for web keys (#1448)
* saving progress

* proto changes

* run make protos

* verify response, add test case

* resolve linter warning about unescaped . in regex pattern

* resolve overlapping proto number
2023-07-18 09:53:12 -04:00
Brandon Yan cab416b533 add launch_darkly keyword to launchdarkly scanner (#1495) 2023-07-17 14:05:58 -05:00
Cody Rose ee814a67bd tweak jdbc redaction (#1490)
JDBC redaction could fail in some irritating edge cases involving passwords that contain the @ character. The logic has been tweaked to eliminate these cases and some tests have been added.
2023-07-17 11:04:12 -04:00
Brandon Yan 17b90b32d0 add couchbase scanner to defaults (#1497)
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 17:45:56 -07:00
Brandon Yan 9af31f00a9 add envoy api key scanner (#1482)
* add envoy api key scanner

* Use detectors4.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 16:46:28 -07:00
trufflesteeeve d03a74776e Fix URI detector false results when the redacted password has been URL encoded (#1489) 2023-07-14 13:35:50 -04:00
Brandon Yan aab8fddc67 fix twilio verification side effect (#1494)
change POST request to GET request
2023-07-13 17:48:51 -07:00
Miccah 4e774d1f01 Define SourceUnit chunking interface (#1484)
* Define SourceUnit chunking interface

* Refactor to use a ChunkReporter interface

* Rename shadowed err to scanErr
2023-07-13 14:11:43 -05:00
Miccah 4b7f94dea1 Rewrite SourceUnitEnumerator to use UnitReporter instead of a channel (#1485) 2023-07-13 13:48:33 -05:00
ahrav a9213a1103 [chore] - Update loop to switch. (#1487)
* Update loop to switch.

* remove unused fxn.
2023-07-12 15:47:43 -07:00
Richard Gomez 1594fddf05 feat(git): include line in github & gitlab links (#1466) 2023-07-11 20:02:27 -07:00
Zachary Rice b48ac24c46 Dedupe results (#1479)
* init 4 dedupin

* use raw rather than rawv2

* rm comment

* comments

* nits

* clean up and use rawv2 too

* add decoder order test
2023-07-11 15:48:00 -05:00
Cody Rose a123d5c5e1 do not report 403s as indeterminate in AWS detector (#1481) 2023-07-11 16:35:23 -04:00
Cody Rose b803a0f701 Report indeterminacy in AWS verifier (#1480) 2023-07-11 15:50:31 -04:00
Zubair Khan 4334af4d34 scan GitHub PR and issue comments (#1435)
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
2023-07-11 15:13:33 -04:00
Miccah 5c0ffda618 Define SourceUnit enumeration interface (#1428)
* Add CancellableWrite helper function

* Create SourceUnitEnumerator interface and EnumerationResult struct

* Implement SourceUnitEnumerator for the filesystem Source

* Omit explicit zero values
2023-07-10 15:05:40 -05:00
Zachary Rice d4972313ff remove old detector (#1474) 2023-07-10 13:02:19 -05:00
Cody Rose 87058dd7fa Add new verification error message field (#1463) 2023-07-10 11:15:40 -04:00
Zachary Rice 0bdd513d88 additional similarity check for base64 and plain (#1462)
* additional similarity check for base64 and plain

* use bytes equal

* move logic into util function
2023-07-10 10:12:59 -05:00
Zubair Khan b38857edb4 fix missing api key, tighten up regex pattern, use response body check (#1438) 2023-07-06 16:35:52 -04:00
Richard Gomez 23757dbe0a remove image4 detector (#1461) 2023-07-06 12:56:09 -07:00
Zachary Rice a99d89d711 fix typo (#1452) 2023-07-05 14:14:18 -05:00
Zachary Rice 8a508e6bcd Add missing keywords for sqlserver (#1449) 2023-07-05 11:12:19 -05:00
Zachary Rice 452734adc8 remove head from git diff command, rename unstaged to staged (#1439) 2023-06-29 15:33:30 -05:00
Zachary Rice 18a70b64bb Introduce trufflehog:ignore tag feature (#1433)
* init ignore

* cleanup and add test

* update readme
2023-06-29 08:45:56 -05:00
roxanne-tampus 00920984e3 added opsgenie detector (#650)
* added opsgenie detector

* update interface and import

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-06-27 16:43:25 -07:00
Zubair Khan d6375ba921 verify response body with expected keywords (#1419)
* verify response body with expected keywords

* remove debug log

* add extra test case

* migrate from ioutil to io

* close body and only check for one keyword

* cleanup
2023-06-27 11:46:15 -04:00
Miccah 8ea49de490 Don't return on okta credential failed verification (#1432) 2023-06-27 09:21:39 -05:00
Zachary Rice 4a77688097 use stringer again for now (#1430) 2023-06-26 14:33:54 -05:00
trufflesteeeve 11bff81def Use url redaction in git (#1399)
Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
2023-06-26 13:56:08 -05:00
Zubair Khan f52946b996 Add Couchbase Detector (#1385)
* init

* add detector type

* rotate leaked credentials

* tighten up username pattern

* isolated prefixregex as overrriding new line stuff

* passwordPat working now

* add username test

* fix edge case

* cleanup

* make linter happy

* make linter happy rd 2

* skip error logging

* fix test

* add password regex helper func

* make test more robust

* cleanup PR

* remove comments

* clarify prepend rationale
2023-06-26 14:37:10 -04:00
Miccah 945c27cb82 Fix docker source to return any chunk errors (#1429) 2023-06-26 12:12:46 -05:00
Brendan Shaklovitz da5301ea1e Exit with non-zero exit code on chunk source error (#1286)
* Exit with non-zero exit code on chunk source error

* Exit with a non-zero exit code whenever we hit an error getting
  chunks. Previously the error would be logged but trufflehog would exit
  with a 0 (success) status code.

* fix gcs test

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-06-26 11:39:57 -05:00
Chris Atkin 6e6895b48e Update Slack webhook error text for verification (#1427)
This updates the matched error text to determine the verified status of a Slack webhook, as this has been updated on Slack's API.
2023-06-26 08:44:17 -07:00
Zubair Khan cd67f6bf16 prevent www from being a key to prevent fp (#1418) 2023-06-25 11:55:11 -04:00
Dustin Decker eeefde1ec9 Ensure results are collected correctly when verification is off, and dedupe twilio (#1420) 2023-06-23 14:14:08 -07:00