a9213a1103
[chore] - Update loop to switch. ( #1487 )
...
* Update loop to switch.
* remove unused fxn.
2023-07-12 15:47:43 -07:00
b48ac24c46
Dedupe results ( #1479 )
...
* init 4 dedupin
* use raw rather than rawv2
* rm comment
* comments
* nits
* clean up and use rawv2 too
* add decoder order test
2023-07-11 15:48:00 -05:00
d4972313ff
remove old detector ( #1474 )
2023-07-10 13:02:19 -05:00
0bdd513d88
additional similarity check for base64 and plain ( #1462 )
...
* additional similarity check for base64 and plain
* use bytes equal
* move logic into util function
2023-07-10 10:12:59 -05:00
23757dbe0a
remove image4 detector ( #1461 )
2023-07-06 12:56:09 -07:00
18a70b64bb
Introduce trufflehog:ignore tag feature ( #1433 )
...
* init ignore
* cleanup and add test
* update readme
2023-06-29 08:45:56 -05:00
00920984e3
added opsgenie detector ( #650 )
...
* added opsgenie detector
* update interface and import
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
2023-06-27 16:43:25 -07:00
945c27cb82
Fix docker source to return any chunk errors ( #1429 )
2023-06-26 12:12:46 -05:00
da5301ea1e
Exit with non-zero exit code on chunk source error ( #1286 )
...
* Exit with non-zero exit code on chunk source error
* Exit with a non-zero exit code whenever we hit an error getting
chunks. Previously the error would be logged but trufflehog would exit
with a 0 (success) status code.
* fix gcs test
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
Co-authored-by: ahrav <ahravdutta02@gmail.com >
2023-06-26 11:39:57 -05:00
e856a6890d
🎉 Add Docker image scanning 🎉 ( #1412 )
...
* Add Docker source
* Add metrics
* Add test
* Add debugging, address PR comments, fix path output
* review suggestions
2023-06-22 08:02:25 -07:00
74ffbd2878
add a custom detector check for logging duplicate detector ( #1394 )
...
* add a custom detector check for logging duplicate detector
* use pb type
2023-06-13 14:49:21 -05:00
dfb1a0cd38
Add DocuSign detector ( #1382 )
...
* init
* look for client id and client secret, encode them for basis auth
* add tests
* test without checking the contents of response
* confirm access_token exists
* cleanup test
* explain in code that an undocumented grant_type is used
* remove use of deprecated ioutil, remove dead code, return errors instead of just logging
* directly pull access token
* update error text, remove redundant body close()
* import new detector into defaults
2023-06-08 13:34:50 -04:00
1da7720912
Replace context.TODO. ( #1349 )
2023-05-19 11:09:51 -07:00
e81b908e07
Add buildkitev2 detector for newer tokens. ( #1341 )
2023-05-15 12:58:36 -07:00
4c04bbbe85
added pulumi cloud Access token detector ( #1295 )
...
* added pulumi cloud Access token detector
* removed accidentally committed tokens
* added the databricks token detection
* made recommended changes
* added supabase management api token
* nuget api key detector
* added aiven.io token detector
* added prefect.io api key detector
* update protos.
---------
Co-authored-by: Developer <garg47294+1@gmail.com >
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com >
2023-05-11 09:08:48 -07:00
584db86031
Support line numbers in filesystem source ( #1297 )
2023-05-09 08:02:34 -07:00
fad34d4dc6
git worktree scanning fix for #827 ( #1315 )
...
* Fix worktree scan by setting EnableDotGitCommonDir
* Change `PlainOpenOptions` to set `EnableDotGitCommonDir` to true.
In every current usage of this function, it is on an already-cloned
repository, so it should always be valid to have this set. By doing
so, it should fix some issues with worktrees.
* Remove unused go.mod replace directives
* Remove replace directives for libraries that are not in use.
2023-05-09 08:00:47 -07:00
c13c56283d
add tineswebhook detector ( #1304 )
2023-05-01 07:48:58 -07:00
b1675194ca
Implement EndpointCustomizer ( #1291 )
...
* Implement EndpointCustomizer
Add the EndpointCustomizer interface and EndpointSetter convenience struct,
implement EndpointCustomizer for github and gitlab detectors, and add
parsing, verification, and applying user-supplied configuration.
* Check error from SetEndpoints
* Rename variable for clarity
2023-04-27 12:23:50 -05:00
5a86c18302
Fix include and exclude detector logic ( #1267 )
...
* Fix include and exclude detector logic
* Fix test
* Add more clarifying comments
2023-04-26 10:49:54 -05:00
6f801f64c7
Added a new detector for percy.io ( #1284 )
...
* Feature: Added a new detector for percy.io
* Updated variable name
---------
Co-authored-by: ahrav <ahravdutta02@gmail.com >
2023-04-25 13:18:34 -07:00
cec1543894
Add utf16 decoder proto. ( #1276 )
2023-04-20 15:25:36 -07:00
dfc5a9f5db
[chore] Log possible duplicate detectors ( #1266 )
...
* [chore] Log possible duplicate detectors
* Fix typos
2023-04-18 10:36:00 -05:00
1c89e79c2d
Remove toLower call on decoded chunk ( #1254 )
...
* remove to lower on decoded data
* clean up
2023-04-14 07:29:32 -05:00
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN ( #1170 )
...
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg
* fix error message
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
2023-04-03 14:56:43 -07:00
0052f60090
Allow for custom verifier ( #1070 )
...
* allow for custom verifier.
* Update engine.
* use custom detectors.
* set cap.
* Update verifiers.
* Remove nil check.
* resolved nit
* handle uppercase values
* updating missing url logs
* adding more descriptive variable names
* updating logs to use correct variables
* Removing toLower for urls
* if else nits
* Adding versioning for github and gitlab
---------
Co-authored-by: ahmed <ahmed.zahran@trufflesec.com >
Co-authored-by: ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com >
2023-03-29 12:26:39 -07:00
fb9ae75661
Support for exclude globs at the git log level ( #1202 )
...
* init
* seems to be working
* better comment
* rm conditional
* Add more context to exclude-globs description
2023-03-28 10:46:03 -05:00
3e4496156c
added new detectors and fixed mesibo detector ( #1166 )
...
* added new detectors and fixed mesibo detector
* added bscscan.com API detector
* added coinmarketcap detector
* update alchemy
* update blocknative
* update bscscan test
* update cmc test
* update tests
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
2023-03-16 18:57:08 -07:00
d382d5cb1c
Add OpenAI API Tokens detector ( #1142 )
...
* Add OpenAI Detector
* Add OpenAI Detector tests
* Add OpenAI Detector to defaults.go
* Removing references to github detector in tests
2023-03-16 17:58:22 -07:00
c617bd7a4e
Add resuming capability to GCS source ( #1161 )
...
* Add resuming capability to GCS source.
* Handle no auth scans.
* complete resume logic
* Use custom function type.
* remove functions.
* linter.
* fix test.
* fix test.
* Handle concurrent map writes.
* use string as CLI flag for include/exclude.
* handle emtpy buckets.
* Handle enumeration on initial job run.
* Rename stats to attributes.
* remove redundant return.
* If test fails due to 400, that is fine, it's expected.
* Add unauth GCS source type.
* comments.
* update proto.
* Use short flag.
* address comments.
2023-03-16 17:53:42 -07:00
6193509098
add support for json service account and service account file. ( #1185 )
2023-03-16 13:04:36 -07:00
f0b6b5d0d9
add a break statement when iterating through keywords ( #1184 )
2023-03-15 16:51:03 -05:00
cbf299aa77
Add gcs scanning integration ( #1153 )
...
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* Remove period from file extension.
* remove used.
* Add comment.
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* remove used.
* Add file type for objects.
* Add check for file type and size.
* Add default file size.
* Add additinoal auth options and remaining CLI flags.
* Handle errors in go routines.
* Handle resuming for buckets.
* Remove redundant words in comment.
* remove ok check on bool check.
* remove extra blank line.
* Add return if handler handles chunk.
* Add comment.
* remove extra blank line.
* cleanup comment.
* Add comment.
* move up fxn.
* go mod tidy.
* Add exclusion to perf testing buckets.
* Handle blocking the channel.
* remove unused const.
* fix tests.
* fix tests.
* Handle gcs manger options better.
* update fxn name.
* Remove arg name.
* ignore buckets in gcsManager test.
* fix test.
* propulate gsManagerOpts.
* inline err check.
* Add readme.
* update readme spelling.
* fix test.
2023-03-07 17:32:04 -08:00
4777b77ec6
Keyword optimization ( #1144 )
...
* init
* ignore trufflehog binary and added comment
* remove unused keywords in chunk, better comment
* remove keywords from engine struct
2023-03-02 11:32:37 -06:00
dd39848709
Add ability to include and exclude detectors ( #1106 )
...
* Add ability to include and exclude detectors
* Trim space before checking for empty items
* Explicitly check for integer overflow
* Use strconv.ParseInt instead of strconv.Atoi
* Address comments
2023-02-27 16:46:45 -06:00
c5b4d6f28b
Support file scanning in filesystem source ( #1030 )
...
* Rename directories to paths
* Generate protos
* Add file scanning support to filesystem source
* Add directories back to filesystem proto
* Generate protos
* Combine paths and directories from in source
* Add filesystem filter
* Address comments
2023-02-27 12:15:05 -06:00
f2139a7615
Github filter support for exclude and include ( #1087 )
...
* test
* Add missing head and base hash back.
---------
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com >
2023-02-14 08:40:53 -08:00
4f13090c01
Remove duplicated detectors ( #1092 )
...
In this case just Heroku and LinearAPI. But this includes the Moonclerck
detector, which appears to be a typo that got turned into a separate
detector type.
Co-authored-by: zubairk14 <zubair.khan@trufflesec.com >
2023-02-13 11:44:19 -05:00
0c66d30c1f
Revert "Make detectors configurable ( #1084 )" ( #1097 )
...
This reverts commit 67784f6928
2023-02-11 08:12:13 -08:00
67784f6928
Make detectors configurable ( #1084 )
...
* Make detectors configurable.
* remove redundant check.
* add number of detectors.
* update comment.
* remove reflect.
* inline key.
* replace name w/ type.
* remove temp var.
* fix test name.
* fix engine start.
* add filter unverified to engine.
* reorder engine args.
* Address comments.
* Add include and exclude.
* update comments.
* add comment.
* add comment.
2023-02-10 16:30:38 -08:00
c5c8d10d28
[chore] - Remove monolithic config struct ( #1091 )
...
* REmove monolithic config struct.
* fix broken test.
2023-02-10 12:43:00 -08:00
d317ddb51a
[chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources ( #1089 )
...
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources
* Address comments
2023-02-10 11:02:55 -06:00
0ce72ccda3
[chore] Remove logrus from github source ( #1086 )
...
* [chore] Remove logrus from github source
* Fix handleRateLimit test
* Fix tests
2023-02-09 18:02:04 -06:00
58e8c1e4ac
[chore] Remove logrus from engine package ( #1085 )
2023-02-09 16:55:19 -06:00
b29b78c10d
filesystem support for exclude and include filters (2nd attemp) ( #1033 )
...
* fix filter issue - empty lines should be ignored
* filesystem support for filter exclude
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
2023-01-26 09:33:45 -08:00
319ae64a02
[chore] - Small cleanup of CircleCi source ( #1028 )
...
* Small cleanup of CircleCi source.
* address comments.
* Add context to methods as first param.
2023-01-17 09:36:18 -08:00
430d5c764c
Rename and export isGitSource ( #1016 )
2023-01-10 12:51:58 -08:00
8b2e1d36cf
Copy metadata for line number aware sources ( #1011 )
...
* Copy metadata for line number aware sources
* Improve style
2023-01-10 09:35:44 -08:00
5f6143f09a
Add Circle CI source ( #997 )
...
* Add Circle CI source
* remove SHA1 line
* remove trim
2023-01-05 21:44:37 -08:00
936a139596
Allow using a glob for include list. ( #977 )
...
* Allow using a glob for include list.
* Update command flag.
* Make comment more clear.
* update comment.
* Allow scanning repo and org at the same time.
2022-12-16 13:28:16 -08:00
ahrav