The previous implementation used int64 for both, which can be mixed up
easily. Using distinct types adds a layer of type safety checked by the
compiler.
This PR implements validation of Gitlab source configuration.
I was hoping to be able to unify more of the implementation of Validate and Chunks, but there was more divergence than I expected. Specifically, Chunks handles a fair number of Gitlab errors that aren't configuration errors (e.g. "Gitlab returned a repo with an unparseable URL"). Accommodating these in the Validate code path felt wrong, and I wasn't able to create a common code path that could accommodate both Validate and Chunks without looking awful.
* Implement CommonSourceUnitUnmarshaller
* Add SourceUnitUnmarshaller to all sources using
All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.
* Fix method comments to adhere to Go's style guide
* Handle errors w/ github source.
* Fix loop var captured by func literal.
* Fix loop var captured by func literal.
* Set completed progress if the scan completes with no errors.
* Set progress to 100% if the scope and iteration are both 0.
* Fix commentary.
* Fix test.
* Return after the defer to os.RemoveAll.
* Fix unauth scan.
* Inline range loop.
* update tests for partial scan completion with errors. Ensure correct progress is set.
* Update progress for all sources.
* Update github test.
* Address comments.
* Fix broken gitlab test.
* Close chunks chan from blocking indefinitely.
* Range over chunksCh in the event chunksCh is nil we don't run into an invalid memory address error.
Update warnings and move clone output information back.
* Remove commented out code.
* Remove .Run() because .CombinedOutput() should call .Run()
* Update test to include count check.
* Address PR comments.
* Fix merge issue.
* Use oauth2 as the username for cloning private repos with a token.
* Update username string.
* Only set user to "placeholder" if no username is present.
* Add `since_commit` to git scan
* Support `max_depth` option for git scan
* Use new options in github and gitlab sources
* Address review feedback
Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
Miccah