Commit Graph

36 Commits

Author SHA1 Message Date
Cody Rose ebf1038392 Support indeterminacy in alchemy and update detector docs (#1510) 2023-07-21 14:50:14 -04:00
Miccah 5c0ffda618 Define SourceUnit enumeration interface (#1428)
* Add CancellableWrite helper function

* Create SourceUnitEnumerator interface and EnumerationResult struct

* Implement SourceUnitEnumerator for the filesystem Source

* Omit explicit zero values
2023-07-10 15:05:40 -05:00
Zachary Rice 0bdd513d88 additional similarity check for base64 and plain (#1462)
* additional similarity check for base64 and plain

* use bytes equal

* move logic into util function
2023-07-10 10:12:59 -05:00
Zubair Khan d6375ba921 verify response body with expected keywords (#1419)
* verify response body with expected keywords

* remove debug log

* add extra test case

* migrate from ioutil to io

* close body and only check for one keyword

* cleanup
2023-06-27 11:46:15 -04:00
Zubair Khan f52946b996 Add Couchbase Detector (#1385)
* init

* add detector type

* rotate leaked credentials

* tighten up username pattern

* isolated prefixregex as overrriding new line stuff

* passwordPat working now

* add username test

* fix edge case

* cleanup

* make linter happy

* make linter happy rd 2

* skip error logging

* fix test

* add password regex helper func

* make test more robust

* cleanup PR

* remove comments

* clarify prepend rationale
2023-06-26 14:37:10 -04:00
Dustin Decker e856a6890d 🎉 Add Docker image scanning 🎉 (#1412)
* Add Docker source

* Add metrics

* Add test

* Add debugging, address PR comments, fix path output

* review suggestions
2023-06-22 08:02:25 -07:00
Bill Rich 6d9ae7acbb Make trace error message so newlines aren't escaped (#1396) 2023-06-14 17:24:31 -07:00
ahrav 5c99a1e754 Remove period from file extension (#1154)
* Remove period from file extension.

* Add comment.
2023-03-06 14:49:16 -08:00
Miccah 4efe5313f4 [chore] Address lint errors (#1133)
* Update strings.Title to cases.Title

* Migrate go-genproto to google-cloud-go

See: https://github.com/googleapis/google-cloud-go/blob/main/migration.md

* Check error in test

* Check error from sem.Acquire

* Remove unused code
2023-02-27 21:03:47 -06:00
Miccah 161e499142 [chore] Remove logrus from trufflehog (#1095)
* [chore] Remove logrus from trufflehog

* Minor fixes

* Fix logFatal call

* Fix logrus call
2023-02-14 17:00:07 -06:00
Alexandr Marchenko b29b78c10d filesystem support for exclude and include filters (2nd attemp) (#1033)
* fix filter issue - empty lines should be ignored

* filesystem support for filter exclude

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-01-26 09:33:45 -08:00
Dustin Decker bc27fef7bc remove logger from retryable client, it is not respecting loglevels (#1020) 2023-01-13 15:28:00 -08:00
Yassine Ilmi d720c0c0f3 Switch to retryableHttpClient for GitHub AuthN API Client + More Logs (#995)
* Adding missing flags to Readme

* Use retryableHttpClient by default for GitHub

* Adding repoUrl for scanning time log

* Use WithField instead of WithFields

* Updating README with lasted --help output
2023-01-09 09:21:56 -08:00
Dustin Decker 28dd25beeb S3 scanner improvements (#938) 2022-11-21 19:15:26 -08:00
Miccah 696f5c68f4 Log the stack trace and recover object (#923)
* Log the stack trace and recover object

* Remove stderr log
2022-11-15 16:48:02 -06:00
Bill Rich 958266ea84 Run chunker in pipeline (#859)
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
ahrav 92f40c2031 [THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
Dustin Decker c12be4d98d Add Retryable HTTP client that suports custom timeout 2022-09-07 18:23:20 -07:00
Dustin Decker fa9479100e Add common sentry recover library and add into goroutines (#738)
* Add common sentry recover library and add into goroutines

* fix nits
2022-08-29 11:45:37 -07:00
Bill Rich af34a6a108 Check for nil filter (#714) 2022-08-15 14:24:19 -07:00
Bill Rich a473b9aa99 Use re-readable reader and common chunker (#703)
* Use re-readable reader and common chunker

* Linter feedback

* Break on error
2022-08-10 15:32:49 -07:00
ahrav dcc102a81c [Thog-371] Utilize config struct for engine scans (#700)
* Use a config struct when scanning and engine source.

* fix tests.

* Move test_helpers to the sources pkg.

* Handle ScanGit error in tests.

* adderss comments.

* Use functional options.

* Remove temp var.

* Add better var names for the setup functions for each config.

* Remove unused var.

* fix error logs.

* fix error logs.

* single line.

* remove blank lines.
2022-08-10 10:11:13 -07:00
Dustin Decker 2d3ddad076 Pr/589 (#654)
* added common regex patterns for detectors

* For HexPattern

* enhancements

* used parseInt

* enhancement

* enhanced regex for email and subdomain

* enhancement for email pattern

* update pattern and detector

Co-authored-by: Roxanne Tampus <roxannetampus02@gmail.com>
2022-07-12 08:34:02 -07:00
Dustin Decker c52545a0d3 Increase default timeout for SaneHttpClient, but fail early. (#587)
* Increase default timeout for SaneHttpClient, but fail early.

* Don't change default for client with timeout
2022-05-26 11:08:38 -07:00
Dustin Decker dd86389487 Address some linting issues (#390) 2022-04-12 22:11:34 -07:00
Dustin Decker d85864a896 Add more contributing docs 2022-04-03 22:48:43 -07:00
Dustin Decker aea937422e Detector improvements (#125)
* New tokens and endpoints  (#115)

Co-authored-by: dmarquero <danee.arquero@gmail.com>
2022-04-03 11:20:09 -07:00
ahrav cedb3393d1 [THOG-128] Code cleanup/ OSS onboarding (#117)
* Small amount of code clean up.

* Rename sem to concurrency for better readability and to remove an extra comment.

* fix stashing issue.

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2022-04-01 16:47:27 -07:00
Dustin Decker c20e9f4732 improvements 2022-03-04 08:39:17 -08:00
Dustin Decker 77418fb3f8 module v3 2022-02-15 18:54:47 -08:00
Bill Rich 2d8756938d Fast git scanning (#40)
* Fast git scanning

* Use original tests

* Use committer time

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
2022-02-15 18:54:47 -08:00
Bill Rich 1fb767247f Add missing pagination on github calls (#30)
* Add missing pagination on github calls

Includes some refactoring to improve readability and code reuse.

* Close response body and handle rate limit

* Re-include support for including users as repos to github scans

* Fix gist test to match new func signature

* Add current test name to logging

* Support username as org use case

* Also include no-auth user as org

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
2022-02-15 18:54:47 -08:00
Dustin Decker 3da3f1ec94 Add gitlab pagination support (#26) 2022-02-15 18:54:47 -08:00
Dustin Decker d6ffadb1ee Initial docs and release automation (#5) 2022-01-18 16:59:18 -08:00
Bill Rich 1422cc303a Include and exclude paths args for gitscan (#6)
* include and exclude paths gitscan args

Add support for include_paths and exclude_paths arguments when scanning
git sources.

* Improve variable name

Co-authored-by: Bill Rich <hrich@Bills-MacBook-Pro.local>
2022-01-14 12:40:50 -08:00
Dustin Decker 4218c39d99 Initial CLI w/ partially implemented Git source and demo detector (#1) 2022-01-13 12:02:24 -08:00