Commit Graph

23 Commits

Author SHA1 Message Date
Brendan Shaklovitz da5301ea1e Exit with non-zero exit code on chunk source error (#1286)
* Exit with non-zero exit code on chunk source error

* Exit with a non-zero exit code whenever we hit an error getting
  chunks. Previously the error would be logged but trufflehog would exit
  with a 0 (success) status code.

* fix gcs test

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-06-26 11:39:57 -05:00
Miccah f3152b6885 Implement SourceUnitUnmarshaller for all sources (#1416)
* Implement CommonSourceUnitUnmarshaller

* Add SourceUnitUnmarshaller to all sources using

All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.

* Fix method comments to adhere to Go's style guide
2023-06-23 11:15:51 -05:00
Brendan Shaklovitz 10902f802a Add max object size flag for s3 bucket scanning (#1294)
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-26 15:39:43 -07:00
iamjpotts b3d917f9c7 Resolve #1167 by adding support for the AWS_SESSION_TOKEN (#1170)
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg

* fix error message

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-03 14:56:43 -07:00
Miccah d317ddb51a [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources (#1089)
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources

* Address comments
2023-02-10 11:02:55 -06:00
ahrav 8be89a593b Handle errors in a thread safe manner (#1052)
* Handle errors in a thread safe manner.

* fix test.

* fix linter.

* address comments.
2023-02-02 11:05:33 -08:00
ahrav 009756dce6 add proto that was missing. (#986) 2022-12-23 13:27:07 -08:00
Bill Rich 36ca2601e0 Add s3 object count to trace logs (#975)
* Add s3 object count to trace logs

* fix debug level
2022-12-13 16:46:09 -08:00
ahrav 26befdd1ec [bug] - Handle error when scanning s3 bucket. (#969)
* Handle error when scanning s# bucket.

* move wait outside loop.

* Add logging.

* revert changes.

* remove.

* revert.
2022-12-12 10:10:06 -08:00
Bill Rich f1ec9e74eb Close files to clean up tmp files (#940) 2022-11-22 13:13:34 -08:00
Dustin Decker 28dd25beeb S3 scanner improvements (#938) 2022-11-21 19:15:26 -08:00
Bill Rich ab71b93f7d Add context to handler (#877)
* Add context to handler

* Return rather than break out of select
2022-10-28 08:57:55 -07:00
Bill Rich 958266ea84 Run chunker in pipeline (#859)
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
ahrav 92f40c2031 [THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
ahrav 7ba583ca40 [THOG-681] - Handle errors sources (#783)
* Handle errors w/ github source.

* Fix loop var captured by func literal.

* Fix loop var captured by func literal.

* Set completed progress if the scan completes with no errors.

* Set progress to 100% if the scope and iteration are both 0.

* Fix commentary.

* Fix test.

* Return after the defer to os.RemoveAll.

* Fix unauth scan.

* Inline range loop.

* update tests for partial scan completion with errors. Ensure correct progress is set.

* Update progress for all sources.

* Update github test.

* Address comments.
2022-09-07 19:40:37 -07:00
Dustin Decker fa9479100e Add common sentry recover library and add into goroutines (#738)
* Add common sentry recover library and add into goroutines

* fix nits
2022-08-29 11:45:37 -07:00
Bill Rich 0ddd49a1b8 Use file handler and common chunker (#707) 2022-08-23 16:35:52 -07:00
ahrav dcc102a81c [Thog-371] Utilize config struct for engine scans (#700)
* Use a config struct when scanning and engine source.

* fix tests.

* Move test_helpers to the sources pkg.

* Handle ScanGit error in tests.

* adderss comments.

* Use functional options.

* Remove temp var.

* Add better var names for the setup functions for each config.

* Remove unused var.

* fix error logs.

* fix error logs.

* single line.

* remove blank lines.
2022-08-10 10:11:13 -07:00
Dustin Decker 2178f1f42e reword and fix error logging 2022-06-13 16:14:22 -07:00
ahrav d2605354fe [THOG-332 ]Remove TokenSource interface from the init method of Source. (#539)
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
2022-05-13 14:35:06 -07:00
ahrav b0d79180f6 [THOG-314] Add new parameter to the Init method for the source interface. (#529)
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
2022-05-10 11:11:43 -07:00
steeeve a770f643df Add placeholder for encoded resume info in SetProgressComplete 2022-03-24 12:43:36 -04:00
Bill Rich 6486c18565 Add s3 support to CLI (#76)
* Add s3 support to CLI

* Clean up comments

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2022-03-14 17:07:07 -07:00