Commit Graph

1827 Commits

Author SHA1 Message Date
Dustin Decker 1e9ed47582 Add panic test case to fix to gitparse 2023-07-28 08:43:33 -07:00
Zubair Khan 9f3809f19e gdrive proto change (#1566) 2023-07-28 10:38:51 -04:00
Miccah e391e89f3e Initial implementation of JobReport with SourceManager usage (#1557)
* Initial implementation of JobReport with SourceManager usage

* Limit concurrent units

* Only save the last JobReport per handle
2023-07-27 10:49:56 -05:00
Zachary Rice 3897454dbb add merge support (#1561) 2023-07-27 09:24:49 -05:00
ahrav ade5d91d5c Add azure repos protos. (#1559) 2023-07-26 19:53:10 -07:00
Richard Gomez f925da7cea fix(mongodb): detect CosmoDB access keys (#1511)
https://learn.microsoft.com/en-us/microsoft-365/compliance/sit-defn-azure-cosmos-db-account-access-key?view=o365-worldwide
2023-07-26 16:50:12 -05:00
Richard Gomez 46823f77c9 feat(github): clarify comment log statement (#1553) 2023-07-26 09:40:30 -05:00
Miccah 10f0963bc9 Add SourceManager tests for Run and Wait methods (#1530)
* Miscellaneous SourceManager updates

* Own the chunks channel instead of accepting it as an input
* Add Chunks and Wait methods
* Fix bug in Enroll so it actually returns the handle
* Add context.Context parameter to the SourceInitFunc type

* Add SourceManager tests for Run and Wait methods

* Rename man variables to mgr
2023-07-26 00:48:28 -05:00
Richard Gomez 2290954b02 fix(github): use apiEndpoint for basic or no auth (#1454) 2023-07-25 20:03:08 -07:00
Richard Gomez f48a635c34 feat: update gitparse logic (#1486) 2023-07-25 17:52:34 -05:00
Miccah 91cbca941a Fix data race in context wrapper library (#1546) 2023-07-25 17:09:36 -05:00
Zachary Rice 1a1977f7e6 case insensitive (#1547) 2023-07-25 17:01:15 -05:00
Bill Rich f39303495a Add commitsScanned metrics (#1533)
* Add commitsScanned metrics

* Just keep commit count
2023-07-25 11:31:01 -07:00
ahrav b5b01d3eba [chore] - optimize chunker (#1535)
* Use chunkbytes that includes the size of peek.

* linter.

* continue.

* add TotalChunkSize const.
2023-07-24 19:30:29 -07:00
Zachary Rice 85f363f093 init (#1538) 2023-07-24 19:09:57 -05:00
Miccah 93c561f324 Add match boundary to okta regular expressions (#1531) 2023-07-24 10:52:50 -05:00
Dustin Decker fab80445d1 continue scanning on detector / decoder panic (#863) 2023-07-24 07:34:43 -07:00
ahrav 9e0a2e9ddd [chore] - Remove password info from log (#1528)
* Remove password info from log.

* update.

* one more.
2023-07-22 20:25:45 -07:00
Miccah 91c5472876 Implement SourceManager basics (#1515)
* Implement SourceManager basics

* Rename identifiers and add a default headlessAPI implementation

* Rewrite to use SourceInitFunc

* Update variable name to accurately reflect its value
2023-07-21 15:20:25 -05:00
Miccah a613bbb979 [chore] Remove parent manipulation in context package (#1525)
The ability to set the parent allowed creating context cycles which
shouldn't be allowed, or at the very least have unintuitive behavior.
2023-07-21 13:51:51 -05:00
Cody Rose ebf1038392 Support indeterminacy in alchemy and update detector docs (#1510) 2023-07-21 14:50:14 -04:00
Cody Rose 06a562688d capture json error (#1509) 2023-07-21 10:44:47 -05:00
Miccah e8b5e3cea3 Revert "[chore] Remove parent setting / getting in Context wrapper (#1516)" (#1519)
This reverts commit 8ec5e4916c.
This commit is somehow causing AWS verification (and possibly others) to
not work.
2023-07-20 23:31:28 -05:00
Miccah 8ec5e4916c [chore] Remove parent setting / getting in Context wrapper (#1516)
* [chore] Remove parent setting / getting in Context wrapper

* Keep the cancellable context from errgroup
2023-07-20 13:33:09 -05:00
Cody Rose 20b7793828 JDBC indeterminacy (#1507)
This PR adds an indeterminacy check to the JDBC verifiers.
2023-07-19 16:57:57 -04:00
Brandon Yan 8fad5fff79 add dockerhub scanner (#1496)
* add dockerhub scanner

* clean

* clean and fix regex logic and tests

* check length of userMatches before access

* Use camelcase.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-19 09:26:28 -07:00
Cody Rose cb1a63a4e2 unify JDBC detector ping logic (#1506)
Previously, the various JDBC detectors would independently try to verify credentials by a process of trying various permutations of candidates one-by-one. The upcoming tri-state verification work will need to add sophistication to this process in the same way for each one, so this PR first combines all of the logic so it can be upgraded in a single spot.
2023-07-19 11:45:56 -04:00
Zubair Khan be549a7287 add thog enterprise detector for web keys (#1448)
* saving progress

* proto changes

* run make protos

* verify response, add test case

* resolve linter warning about unescaped . in regex pattern

* resolve overlapping proto number
2023-07-18 09:53:12 -04:00
Brandon Yan cab416b533 add launch_darkly keyword to launchdarkly scanner (#1495) 2023-07-17 14:05:58 -05:00
Cody Rose ee814a67bd tweak jdbc redaction (#1490)
JDBC redaction could fail in some irritating edge cases involving passwords that contain the @ character. The logic has been tweaked to eliminate these cases and some tests have been added.
2023-07-17 11:04:12 -04:00
Brandon Yan 17b90b32d0 add couchbase scanner to defaults (#1497)
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 17:45:56 -07:00
Brandon Yan 9af31f00a9 add envoy api key scanner (#1482)
* add envoy api key scanner

* Use detectors4.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 16:46:28 -07:00
trufflesteeeve d03a74776e Fix URI detector false results when the redacted password has been URL encoded (#1489) 2023-07-14 13:35:50 -04:00
Brandon Yan aab8fddc67 fix twilio verification side effect (#1494)
change POST request to GET request
2023-07-13 17:48:51 -07:00
Miccah 4e774d1f01 Define SourceUnit chunking interface (#1484)
* Define SourceUnit chunking interface

* Refactor to use a ChunkReporter interface

* Rename shadowed err to scanErr
2023-07-13 14:11:43 -05:00
Miccah 4b7f94dea1 Rewrite SourceUnitEnumerator to use UnitReporter instead of a channel (#1485) 2023-07-13 13:48:33 -05:00
ahrav a9213a1103 [chore] - Update loop to switch. (#1487)
* Update loop to switch.

* remove unused fxn.
2023-07-12 15:47:43 -07:00
Richard Gomez 1594fddf05 feat(git): include line in github & gitlab links (#1466) 2023-07-11 20:02:27 -07:00
Zachary Rice b48ac24c46 Dedupe results (#1479)
* init 4 dedupin

* use raw rather than rawv2

* rm comment

* comments

* nits

* clean up and use rawv2 too

* add decoder order test
2023-07-11 15:48:00 -05:00
Cody Rose a123d5c5e1 do not report 403s as indeterminate in AWS detector (#1481) 2023-07-11 16:35:23 -04:00
Cody Rose b803a0f701 Report indeterminacy in AWS verifier (#1480) 2023-07-11 15:50:31 -04:00
Zubair Khan 4334af4d34 scan GitHub PR and issue comments (#1435)
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
2023-07-11 15:13:33 -04:00
Miccah 5c0ffda618 Define SourceUnit enumeration interface (#1428)
* Add CancellableWrite helper function

* Create SourceUnitEnumerator interface and EnumerationResult struct

* Implement SourceUnitEnumerator for the filesystem Source

* Omit explicit zero values
2023-07-10 15:05:40 -05:00
Zachary Rice d4972313ff remove old detector (#1474) 2023-07-10 13:02:19 -05:00
Cody Rose 87058dd7fa Add new verification error message field (#1463) 2023-07-10 11:15:40 -04:00
Zachary Rice 0bdd513d88 additional similarity check for base64 and plain (#1462)
* additional similarity check for base64 and plain

* use bytes equal

* move logic into util function
2023-07-10 10:12:59 -05:00
Zubair Khan b38857edb4 fix missing api key, tighten up regex pattern, use response body check (#1438) 2023-07-06 16:35:52 -04:00
Richard Gomez 23757dbe0a remove image4 detector (#1461) 2023-07-06 12:56:09 -07:00
Zachary Rice a99d89d711 fix typo (#1452) 2023-07-05 14:14:18 -05:00
Zachary Rice 8a508e6bcd Add missing keywords for sqlserver (#1449) 2023-07-05 11:12:19 -05:00