4e774d1f01
Define SourceUnit chunking interface ( #1484 )
...
* Define SourceUnit chunking interface
* Refactor to use a ChunkReporter interface
* Rename shadowed err to scanErr
2023-07-13 14:11:43 -05:00
4b7f94dea1
Rewrite SourceUnitEnumerator to use UnitReporter instead of a channel ( #1485 )
2023-07-13 13:48:33 -05:00
5c0ffda618
Define SourceUnit enumeration interface ( #1428 )
...
* Add CancellableWrite helper function
* Create SourceUnitEnumerator interface and EnumerationResult struct
* Implement SourceUnitEnumerator for the filesystem Source
* Omit explicit zero values
2023-07-10 15:05:40 -05:00
f3152b6885
Implement SourceUnitUnmarshaller for all sources ( #1416 )
...
* Implement CommonSourceUnitUnmarshaller
* Add SourceUnitUnmarshaller to all sources using
All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.
* Fix method comments to adhere to Go's style guide
2023-06-23 11:15:51 -05:00
c2e3e7d53a
Split files instead of using ReadAll ( #1387 )
...
* Split files instead of using ReadAll
* Remove dup chunk
* Actually break out of loop
2023-06-12 14:09:05 -07:00
c5b4d6f28b
Support file scanning in filesystem source ( #1030 )
...
* Rename directories to paths
* Generate protos
* Add file scanning support to filesystem source
* Add directories back to filesystem proto
* Generate protos
* Combine paths and directories from in source
* Add filesystem filter
* Address comments
2023-02-27 12:15:05 -06:00
d317ddb51a
[chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources ( #1089 )
...
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources
* Address comments
2023-02-10 11:02:55 -06:00
b29b78c10d
filesystem support for exclude and include filters (2nd attemp) ( #1033 )
...
* fix filter issue - empty lines should be ignored
* filesystem support for filter exclude
Co-authored-by: Dustin Decker <dustin@trufflesec.com >
2023-01-26 09:33:45 -08:00
009756dce6
add proto that was missing. ( #986 )
2022-12-23 13:27:07 -08:00
ab71b93f7d
Add context to handler ( #877 )
...
* Add context to handler
* Return rather than break out of select
2022-10-28 08:57:55 -07:00
958266ea84
Run chunker in pipeline ( #859 )
...
* Run chunker in pipeline
* Move ChunkSize and PeekSize to source package.
* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
785cead43e
Ignore URIs where the password is redacted ( #842 )
...
Only `*`s in the password is a redacted basic auth URI.
2022-10-11 14:18:52 -07:00
d11ce27f33
Use correct reader in filesystem source ( #756 )
2022-08-30 10:24:52 -07:00
fa9479100e
Add common sentry recover library and add into goroutines ( #738 )
...
* Add common sentry recover library and add into goroutines
* fix nits
2022-08-29 11:45:37 -07:00
a473b9aa99
Use re-readable reader and common chunker ( #703 )
...
* Use re-readable reader and common chunker
* Linter feedback
* Break on error
2022-08-10 15:32:49 -07:00
dcc102a81c
[Thog-371] Utilize config struct for engine scans ( #700 )
...
* Use a config struct when scanning and engine source.
* fix tests.
* Move test_helpers to the sources pkg.
* Handle ScanGit error in tests.
* adderss comments.
* Use functional options.
* Remove temp var.
* Add better var names for the setup functions for each config.
* Remove unused var.
* fix error logs.
* fix error logs.
* single line.
* remove blank lines.
2022-08-10 10:11:13 -07:00
30ebe84e3e
[THOG-608] - Fix linter errors. ( #701 )
...
* Fix linter errors.
* Fix gist adding test.
* Update test string for mock JSON reply.
* Remove if.
2022-08-09 19:20:02 -07:00
7273dc9058
Archive decoder ( #683 )
...
* Archive decoder
* Fix reader handling
* Seek error handling
* Add tests
* Fix extra empty chunk
* Sync chunk size
2022-08-02 20:36:21 -07:00
d2605354fe
[THOG-332 ]Remove TokenSource interface from the init method of Source. ( #539 )
...
* Remove TokenSource interface from the init method of Source.
* Remove proto message.
* Remove proto message.
* Fix tests.
* Fix filesystem test.
2022-05-13 14:35:06 -07:00
b0d79180f6
[THOG-314] Add new parameter to the Init method for the source interface. ( #529 )
...
* Add new parameter to the Init method for the source interface.
* Add Oauth Token service.
* remove .test file.
* remove .test file.
* Fix param spelling.
* fix tests with new param in init
* Add missing gock lib.
2022-05-10 11:11:43 -07:00
d78c929385
Actually skip file ( #299 )
2022-04-06 09:48:40 -07:00
33aa6f9cab
Log error and skip file when stat fails ( #296 )
2022-04-05 18:58:05 -07:00
a770f643df
Add placeholder for encoded resume info in SetProgressComplete
2022-03-24 12:43:36 -04:00
5ab5c6f9d9
Only scan regular files ( #87 )
...
* Only scan regular files
* Remove IsDirectory func
2022-03-16 16:04:10 -07:00
c20e9f4732
improvements
2022-03-04 08:39:17 -08:00
77418fb3f8
module v3
2022-02-15 18:54:47 -08:00
4218c39d99
Initial CLI w/ partially implemented Git source and demo detector ( #1 )
2022-01-13 12:02:24 -08:00
Miccah