It turns out that GetCallerIdentity returns a surprising quantity of transient, false-negative 403 responses that carry the SignatureDoesNotMatch error reason. I don't know why this is happening, but their transient nature makes them indeterminate verification failures and they should be flagged as such. The AWS detector has therefore been modified to specifically look for the InvalidClientTokenId error reason in 403 responses and mark all other responses as indeterminate.
In addition to the functional changes this PR contains some updates to the test code that allow us to test them.
This PR makes the Alchemy detector run its known false positive check even if verification is disabled. This isn't the most important detector but it's the template for new ones so getting a good pattern nailed down is important.
Moving the check allowed me to rewrite the determinacy logic to hopefully be more clear.
Previously, the various JDBC detectors would independently try to verify credentials by a process of trying various permutations of candidates one-by-one. The upcoming tri-state verification work will need to add sophistication to this process in the same way for each one, so this PR first combines all of the logic so it can be upgraded in a single spot.
* saving progress
* proto changes
* run make protos
* verify response, add test case
* resolve linter warning about unescaped . in regex pattern
* resolve overlapping proto number
JDBC redaction could fail in some irritating edge cases involving passwords that contain the @ character. The logic has been tweaked to eliminate these cases and some tests have been added.
* verify response body with expected keywords
* remove debug log
* add extra test case
* migrate from ioutil to io
* close body and only check for one keyword
* cleanup
* init
* add detector type
* rotate leaked credentials
* tighten up username pattern
* isolated prefixregex as overrriding new line stuff
* passwordPat working now
* add username test
* fix edge case
* cleanup
* make linter happy
* make linter happy rd 2
* skip error logging
* fix test
* add password regex helper func
* make test more robust
* cleanup PR
* remove comments
* clarify prepend rationale
* init
* look for client id and client secret, encode them for basis auth
* add tests
* test without checking the contents of response
* confirm access_token exists
* cleanup test
* explain in code that an undocumented grant_type is used
* remove use of deprecated ioutil, remove dead code, return errors instead of just logging
* directly pull access token
* update error text, remove redundant body close()
* import new detector into defaults
When a Result is emitted, it should include
the `chunk.Data []byte` so that we can utilize
the blob of data which caused the result.
This makes it so something catching the results
does not have to maintain a collection of chunks
to correlate the two together.
Generic API keys like shown in the example below is getting excluded:
api_key=9e107d9d372bb6826bd81d3542a419d6 because of following regex patterns:
\b[A-Fa-f0-9]{32}\b
\b[A-Fa-f0-9x]{6,99}\b
The base64 decoding logic is getting hit and NOT returning an error, and thus it continues thinking it is base64 decoded.
ahrav