38 Commits

Author SHA1 Message Date
Richard Gomez 1594fddf05 feat(git): include line in github & gitlab links (#1466) 2023-07-11 20:02:27 -07:00
Miccah f3152b6885 Implement SourceUnitUnmarshaller for all sources (#1416)
* Implement CommonSourceUnitUnmarshaller

* Add SourceUnitUnmarshaller to all sources using

All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.

* Fix method comments to adhere to Go's style guide
2023-06-23 11:15:51 -05:00
Dustin Decker c8944825de Surface missing git as an error during initialization (#1362) 2023-05-26 15:23:08 -07:00
Miccah d317ddb51a [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources (#1089)
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources

* Address comments
2023-02-10 11:02:55 -06:00
ahrav e47cc2451f Dont pre-allocate errors slice. (#1083) 2023-02-08 17:33:30 -08:00
ahrav 8be89a593b Handle errors in a thread safe manner (#1052)
* Handle errors in a thread safe manner.

* fix test.

* fix linter.

* address comments.
2023-02-02 11:05:33 -08:00
ahrav 009756dce6 add proto that was missing. (#986) 2022-12-23 13:27:07 -08:00
Dustin Decker 7de9bdd12d Support globbing with ignore repos (#967) 2022-12-09 12:10:42 -08:00
Jessica 3d501975e4 Add filter as scan option to gitlab module's git scan (#919) 2022-11-15 13:02:37 -08:00
ahrav dd141fb55f [oc-147] - Add context to all git methods (#901)
* Add context to all git methods.

* remove logrus.

* Add ctx.

* Address comments.

* Add error to clone failing.

* Return error.
2022-11-03 16:36:52 -07:00
ahrav 029519eb01 [THOG-767] ignore gitlab repos (#853)
* Add ability to ignore repos.

* use std library slices.Contains.

* Add tests.

* Remove zero values from test.
2022-10-19 13:55:44 -07:00
ahrav 7ba583ca40 [THOG-681] - Handle errors sources (#783)
* Handle errors w/ github source.

* Fix loop var captured by func literal.

* Fix loop var captured by func literal.

* Set completed progress if the scan completes with no errors.

* Set progress to 100% if the scope and iteration are both 0.

* Fix commentary.

* Fix test.

* Return after the defer to os.RemoveAll.

* Fix unauth scan.

* Inline range loop.

* update tests for partial scan completion with errors. Ensure correct progress is set.

* Update progress for all sources.

* Update github test.

* Address comments.
2022-09-07 19:40:37 -07:00
Dustin Decker fa9479100e Add common sentry recover library and add into goroutines (#738)
* Add common sentry recover library and add into goroutines

* fix nits
2022-08-29 11:45:37 -07:00
trufflesteeeve 10f4d02c31 Allow gitlab to resume from encoded resume info (#611) 2022-06-17 11:45:17 -04:00
ahrav 2051fe14ff remove profililing. (#567) 2022-05-23 11:05:39 -07:00
ahrav d2605354fe [THOG-332 ]Remove TokenSource interface from the init method of Source. (#539)
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
2022-05-13 14:35:06 -07:00
ahrav b0d79180f6 [THOG-314] Add new parameter to the Init method for the source interface. (#529)
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
2022-05-10 11:11:43 -07:00
Miccah 8cd0831f77 Test Gitlab basic authentication with password and personal access token (#474) 2022-04-23 17:55:13 -07:00
ahrav 753f116c89 Fix broken test for gitlab. (#473) 2022-04-22 10:28:17 -07:00
ahrav 865bc5baf6 [THOG-248] fix broken gitlab tests (#437)
* Fix broken gitlab test.

* Close chunks chan from blocking indefinitely.

* Range over chunksCh in the event chunksCh is nil we don't run into an invalid memory address error.
Update warnings and move clone output information back.

* Remove commented out code.

* Remove .Run() because .CombinedOutput() should call .Run()

* Update test to include count check.

* Address PR comments.

* Fix merge issue.
2022-04-21 18:18:07 -07:00
ahrav 3b35404238 [THOG-204] Use oauth2 as username when cloning (#441)
* Use oauth2 as the username for cloning private repos with a token.

* Update username string.

* Only set user to "placeholder" if no username is present.
2022-04-21 17:49:08 -07:00
dependabot[bot] 5b6c9f7b31 Bump github.com/xanzy/go-gitlab from 0.60.0 to 0.63.0 (#382)
* Bump github.com/xanzy/go-gitlab from 0.60.0 to 0.63.0

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.60.0 to 0.63.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.60.0...v0.63.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix name for updated dependency

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bill-rich <bill.rich@gmail.com>
2022-04-20 23:15:20 -07:00
Dustin Decker ba6ea9d42f Improve error logging and continue when there are insufficient permissions (#315) 2022-04-07 18:33:44 -07:00
Bill Rich 6ed01500f8 Only add trailing slash when endpoint populated (#120) 2022-04-01 11:47:59 -07:00
Bill Rich 155566bbd5 make sure url ends with slash (#110) 2022-03-28 09:56:44 -07:00
steeeve a770f643df Add placeholder for encoded resume info in SetProgressComplete 2022-03-24 12:43:36 -04:00
Bill Rich c2b4f0bc39 Include line numbers in git metadata (#97)
* Include line numbers in git metadata

* Update tests for fragments and line numbers
2022-03-22 09:27:15 -07:00
Bill Rich 0ee34a5be6 Use gitleaks git cmd stream (#75)
* Switch to git cmd stream

* Fix rebase issues
2022-03-14 17:12:58 -07:00
Bill Rich 665b0bf928 Add timestamp (#61)
* Add timestamp to sources

* Include timestamp in sources
2022-03-04 08:39:23 -08:00
Dustin Decker c80bd5e905 Fix linting and dogfood in CI 2022-03-04 08:39:23 -08:00
trufflesteeeve 9cb99e5aa4 578 Fix gitlab basic auth with access tokens (#54) 2022-03-04 08:39:22 -08:00
Dustin Decker c20e9f4732 improvements 2022-03-04 08:39:17 -08:00
Dustin Decker 77418fb3f8 module v3 2022-02-15 18:54:47 -08:00
Dustin Decker 44d113c408 Add concurrency to gitlab source integration 2022-02-15 18:54:47 -08:00
Dustin Decker 3da3f1ec94 Add gitlab pagination support (#26) 2022-02-15 18:54:47 -08:00
Bill Rich 28ed0c3b7c Complete support for existing git scan flags (#13)
* Add `since_commit` to git scan

* Support `max_depth` option for git scan

* Use new options in github and gitlab sources

* Address review feedback

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
2022-02-15 18:54:47 -08:00
Dustin Decker d6ffadb1ee Initial docs and release automation (#5) 2022-01-18 16:59:18 -08:00
Dustin Decker 4218c39d99 Initial CLI w/ partially implemented Git source and demo detector (#1) 2022-01-13 12:02:24 -08:00