Commit Graph

146 Commits

Author SHA1 Message Date
ahrav c5c8d10d28 [chore] - Remove monolithic config struct (#1091)
* REmove monolithic config struct.

* fix broken test.
2023-02-10 12:43:00 -08:00
Miccah d317ddb51a [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources (#1089)
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources

* Address comments
2023-02-10 11:02:55 -06:00
Miccah 0ce72ccda3 [chore] Remove logrus from github source (#1086)
* [chore] Remove logrus from github source

* Fix handleRateLimit test

* Fix tests
2023-02-09 18:02:04 -06:00
Miccah 58e8c1e4ac [chore] Remove logrus from engine package (#1085) 2023-02-09 16:55:19 -06:00
Alexandr Marchenko b29b78c10d filesystem support for exclude and include filters (2nd attemp) (#1033)
* fix filter issue - empty lines should be ignored

* filesystem support for filter exclude

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-01-26 09:33:45 -08:00
ahrav 319ae64a02 [chore] - Small cleanup of CircleCi source (#1028)
* Small cleanup of CircleCi source.

* address comments.

* Add context to methods as first param.
2023-01-17 09:36:18 -08:00
Bill Rich 430d5c764c Rename and export isGitSource (#1016) 2023-01-10 12:51:58 -08:00
Bill Rich 8b2e1d36cf Copy metadata for line number aware sources (#1011)
* Copy metadata for line number aware sources

* Improve style
2023-01-10 09:35:44 -08:00
Dustin Decker 5f6143f09a Add Circle CI source (#997)
* Add Circle CI source

* remove SHA1 line

* remove trim
2023-01-05 21:44:37 -08:00
ahrav 936a139596 Allow using a glob for include list. (#977)
* Allow using a glob for include list.

* Update command flag.

* Make comment more clear.

* update comment.

* Allow scanning repo and org at the same time.
2022-12-16 13:28:16 -08:00
Bill Rich 335ce85ce4 Export line number code (#962) 2022-12-06 15:31:15 -08:00
Thiago Lages de Alencar 9757c339d9 Fix error message (#933) 2022-11-20 05:31:11 -08:00
Dustin Decker ae4b387448 add LDAP detector (#896) 2022-11-18 19:45:11 -08:00
Dustin Decker b18edef01a Enable skipping of particular key IDs (#930)
* Enable skipping of particular key IDs

* update test
2022-11-18 09:09:40 -08:00
Jessica 3d501975e4 Add filter as scan option to gitlab module's git scan (#919) 2022-11-15 13:02:37 -08:00
Bill Rich d3b24fa592 Replace plain decoder with utf8 (#922) 2022-11-15 09:36:01 -08:00
kstilwell ecd25784f5 Adding Shopify detector (#875)
* Fixes/work based on testing

* Remove some commented code

* Change how verification happens and grab additional information

* Address linter warnings.

* add shopify detector to default detectors.

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2022-11-08 16:21:57 -08:00
ahrav dd141fb55f [oc-147] - Add context to all git methods (#901)
* Add context to all git methods.

* remove logrus.

* Add ctx.

* Address comments.

* Add error to clone failing.

* Return error.
2022-11-03 16:36:52 -07:00
Dustin Decker a7fc12240f Do local URI verification, while attempting to defuse SSRF (#879)
* simplify monogo pattern

* do URI verification locally, while attempting to defuse SSRF

* test SSRF defuse

* simplify err check logic per linter recommendation

* split up detectors

* address comments

* remove unused var
2022-11-01 17:27:24 -07:00
ahrav fe029b1098 [THOG-793] - Return all unverified results (#856)
* Remove the check to filter and return only a single unverified result.

* Revert "Remove the check to filter and return only a single unverified result."

This reverts commit 494e432803.

* Add new CLI flag to filter unverified results.
2022-10-31 09:36:10 -07:00
Bill Rich 034ca4fb5b Add bytes counter to scans (#876) 2022-10-27 12:54:22 -07:00
Alexandr Marchenko 60464da3ce proposal: SqlServer connection string detector (#867)
* sqlserver added to detectors.proto

* make protos

* boilerplate detector generated

* wireup

* initial
2022-10-26 07:46:13 -07:00
Bill Rich 958266ea84 Run chunker in pipeline (#859)
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
Ankush Goel d29357c9d4 added npm detector (#841) 2022-10-13 06:04:02 -07:00
Dustin Decker 85467538f6 remove faulty detector (#836) 2022-10-07 09:20:44 -07:00
ahrav 128002885a Add decoder type to results. (#835) 2022-10-06 11:55:07 -07:00
Miccah 2bc4985061 Add SSH config option for the git source (#830)
* Add SSH config option for the git source

The auth message is empty since we use the git binary underneath to
handle the SSH authentication.

* Import digitaloceanv2
2022-09-28 20:40:01 +02:00
Mildred Bernardo ad4b9406a7 Added digitaloceanv2 detector (#829)
* Added digitaloceanv2 detector

* import detector

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2022-09-28 09:56:35 -07:00
ahrav db42bcf2a2 [OC-103] - Add Gemini detector (#800)
* Add Gemini detector.

* Add regex and test code for Gemini detector.

* Remove else.

* Add commentary.

* Address comments.

* Use regular else.

* Make nice and complicated.

* use regular detection pattern.

* Add detector to default detectors.
2022-09-26 11:48:48 -07:00
Bill Rich e3107ad6bb Move head and base normalization to source (#818) 2022-09-23 08:58:45 -07:00
ahrav f2f5b6dba8 Fix recover. (#817) 2022-09-22 15:47:42 -07:00
ahrav 92f40c2031 [THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
Dustin Decker e8f1bb9127 Import NGC detector 2022-09-21 16:26:58 -07:00
Bill Rich 509cf8b6fa Use headref and check empty commits for base (#815) 2022-09-21 16:04:01 -07:00
Dustin Decker fcd580406e A few improvements (#809)
* Run integration tests

* Update examples

* Import mongodb
2022-09-19 13:23:25 -07:00
Dustin Decker fa9479100e Add common sentry recover library and add into goroutines (#738)
* Add common sentry recover library and add into goroutines

* fix nits
2022-08-29 11:45:37 -07:00
Dustin Decker 2452e93a80 Import 27 new detectors (#737) 2022-08-26 12:35:06 -07:00
ahrav 73f9d3f0a0 [chore] - Use config struct instead of pointer for engine scans. (#709)
* Use a config struct instead of pointer when scanning engine sources.

* use config.
2022-08-12 09:56:24 -07:00
ahrav dcc102a81c [Thog-371] Utilize config struct for engine scans (#700)
* Use a config struct when scanning and engine source.

* fix tests.

* Move test_helpers to the sources pkg.

* Handle ScanGit error in tests.

* adderss comments.

* Use functional options.

* Remove temp var.

* Add better var names for the setup functions for each config.

* Remove unused var.

* fix error logs.

* fix error logs.

* single line.

* remove blank lines.
2022-08-10 10:11:13 -07:00
ahrav 30ebe84e3e [THOG-608] - Fix linter errors. (#701)
* Fix linter errors.

* Fix gist adding test.

* Update test string for mock JSON reply.

* Remove if.
2022-08-09 19:20:02 -07:00
trufflesteeeve 176552b07a Fix commit attribution, git tests, and run make protos (#667)
* Update dependency to fix commit attribution, fix git tests

* Run make protos to match code with current proto definitions
2022-07-25 11:44:15 -04:00
Dustin Decker c2426df4d6 Disable sentiment verifier and update teams webhook to avoid being stateful (#646) 2022-07-07 16:55:45 -07:00
Miccah 6fa2171a22 Refactor Engine to wait for workers in a Finish method (#581)
* Refactor Engine to wait for workers in a Finish method

This should allow the engine to run multiple concurrent scans if
desired before shutting down.

Additionally, this commit refactors some of the printing logic to the
output package.

* Fix tests
2022-05-25 11:35:44 -05:00
Dustin Decker 235a60a850 add more detectors (#577) 2022-05-23 18:39:49 -07:00
Dustin Decker 606dadfbb1 Import more detectors (#563) 2022-05-17 18:56:48 -07:00
ahrav d2605354fe [THOG-332 ]Remove TokenSource interface from the init method of Source. (#539)
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
2022-05-13 14:35:06 -07:00
ahrav b0d79180f6 [THOG-314] Add new parameter to the Init method for the source interface. (#529)
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
2022-05-10 11:11:43 -07:00
ahrav e12432cef8 [THOG-315] Replace bytes.buffer with strings.builder. (#533)
* Replace bytes.buffer with string.builder.

* Remove profiling.

* Remove detector changes.

* ignore .test files.

* fix detectors removed.
2022-05-09 17:02:46 -07:00
Bill Rich 6ee5c57968 Fix overlooked nil (#523)
* Fix overlooked nil

* Use default case
2022-05-04 15:45:12 -07:00
Bill Rich c78120e56f Syslog source (#500)
* Add syslog source

* only load cert/key with tls

* Cleanup

* Linting

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
2022-05-04 15:08:11 -07:00