User Guide
Synapse-GitHub User Guide
Synapse-GitHub adds new Storm commands to allow you to query the GitHub API using your existing API key.
Getting Started
Check with your Admin to enable permissions and find out if you need a personal API key.
Examples
Setting your personal API key
To set-up a personal use API key:
> github.setup.apikey --self myapikey
Setting GitHub API key for the current user.
Ingest repositories owned by an account
Ingest the repositories owned by the “vertexproject” account:
> github.repos.pull vertexproject --yield --size 5
WARNING: The property it:dev:repo:created is deprecated or using a deprecated type and will be removed in 3.0.0
it:dev:repo=dbfc7e21bd54adb324521c64dfaee15a
.created = 2024/12/20 18:02:48.407
:created = 2015/06/10 23:29:41.000
:desc = Synapse Central Intelligence System
:name = vertexproject/synapse
:type = git
:url = https://github.com/vertexproject/synapse
it:dev:repo=cb82b5fbad43c019eac4c959cbb580da
.created = 2024/12/20 18:02:48.439
:created = 2017/02/02 17:25:36.000
:desc = Replaced by https://github.com/vertexproject/synapse-base-image
:name = vertexproject/docker-synapse
:type = git
:url = https://github.com/vertexproject/docker-synapse
it:dev:repo=2a242f80b6ec4e9428a52fb591fbc746
.created = 2024/12/20 18:02:48.466
:created = 2017/06/03 18:22:23.000
:desc = Replaced by https://github.com/vertexproject/synapse-base-image
:name = vertexproject/synapse-docker-testimages
:type = git
:url = https://github.com/vertexproject/synapse-docker-testimages
it:dev:repo=2a449fe7b489832678de031f9f1094fb
.created = 2024/12/20 18:02:48.493
:created = 2017/11/08 22:47:50.000
:name = vertexproject/anguine
:type = git
:url = https://github.com/vertexproject/anguine
it:dev:repo=85b2436638199485c236c0a08c9bbaeb
.created = 2024/12/20 18:02:48.520
:created = 2017/11/18 21:47:56.000
:desc = Base images for Synapse testing and the official Synapse Docker containers.
:name = vertexproject/synapse-base-image
:type = git
:url = https://github.com/vertexproject/synapse-base-image
Ingest issues associated with a GitHub repository
Ingest only the closed issues for the vertexproject/synapse repository:
> it:dev:repo:name=vertexproject/synapse | github.issues.pull --yield --size 5 --state closed
WARNING: The property it:dev:repo:issue:created is deprecated or using a deprecated type and will be removed in 3.0.0
WARNING: The property it:dev:repo:issue:label:applied is deprecated or using a deprecated type and will be removed in 3.0.0
it:dev:repo:issue=f353656f58ea2d4742df7971a24c1166
.created = 2024/12/20 18:02:49.033
:created = 2022/11/10 19:10:40.000
:desc = Are there any plans to add type annotations?
Using [`CoreApi.storm()`](https://synapse.docs.vertex.link/en/latest/_modules/synapse/cortex.html#CoreApi.storm) as an `AsyncGenerator`, for instance, has type checkers angry with me.
:id = 2912
:repo = dbfc7e21bd54adb324521c64dfaee15a
:title = type annotations
:updated = 2022/11/12 08:21:44.000
:url = https://github.com/vertexproject/synapse/issues/2912
it:dev:repo:issue=722fc4451d8266dd1814794badd4326d
.created = 2024/12/20 18:02:49.253
:created = 2022/09/28 21:54:54.000
:desc = Similar to what is implemented in csvtool, I would like to request the ability to invoke `cmdr` and/or `storm` shells providing the view to work in as an argument. As previously implemented in csvtool: https://github.com/vertexproject/synapse/pull/2726
Currently the only way, with open source synapse (non-Optic), to work within a view is from within cmdr via `storm --optsfile <path> <query>` This is cumbersome and also error prone as it's easy to woopsie and forget to add your optsfile path resulting in an edit to the master layer and possibly loud shouting.
A bonus may be to create a new view if no view guid is provided as an argument, `--view <guid>` vs. `--view`, prompting the user if this is what they wish to achieve.
:id = 2860
:repo = dbfc7e21bd54adb324521c64dfaee15a
:title = [feature] storm & cmdr - add view and optsfile arguments
:updated = 2023/03/01 23:33:07.000
:url = https://github.com/vertexproject/synapse/issues/2860
it:dev:repo:issue=8a6c6b4d12b4df1546fb75b0e9ecdc04
.created = 2024/12/20 18:02:49.396
:created = 2022/08/30 20:38:47.000
:desc = skimming https://github.com/vertexproject/synapse/blob/master/synapse/tools/csvtool.py -- i'm wondering if csvtool could (in theory) be re-factored to support the REST API ? (ie: are there some specific things telepath provides the REST API does not? or is it simply that it was built for the lower level API and no one has contributed adaptation to interact with the REST API .. yet?)
not asking for work to be done; just getting a sense of how deep that hole might be..
(problem: I have REST api users who may want to use this and other tools, trying to determine future feasibility via the REST API vs telepath (certs, etc)).
:id = 2821
:repo = dbfc7e21bd54adb324521c64dfaee15a
:title = question: could csvtool work (in theory) with the rest api?
:updated = 2023/05/22 11:31:13.000
:url = https://github.com/vertexproject/synapse/issues/2821
it:dev:repo:issue=ea4d976015e13dbe80ac70ed7d2922c6
.created = 2024/12/20 18:02:49.608
:created = 2022/03/16 16:29:03.000
:desc = **Describe the bug**
When updating synapse versions the hugenum change seems to have impacted revCoreLayers in a way that causes cortex to exit. Just checking if this is a known bug (or I may have missed a needed step during an upgrade. Also I am running docker containers against master which is clearly not a best practice). There is no impact for me. I can delete the data volumes and start from scratch, but if I had vital data it would be nice to know how to recover.
**To Reproduce**
docker-compose pull
docker-compose up
version: '3'
services:
cortex_0:
environment:
- SYN_LOG_LEVEL=DEBUG
- SYN_LOG_STRUCT=1
- SYN_CORTEX_AUTH_PASSWD=redacted
- SYN_CORTEX_AXON=tcp://user:redacted@axon_0:27492/
- SYN_CORTEX_NEXSLOG_EN=True
- SYN_CORTEX_PROVENANCE_EN=True
- SYN_CORTEX_STORM_LOG=True
- SYN_CORTEX_STORM_LOG_LEVEL=DEBUG
image: vertexproject/synapse-cortex:master
ports:
- 444:4443
- 27494:27492
volumes:
- ./cortex_0:/vertex/storage
- ./cortex_0:/vertex/backups
-
**Expected behavior**
Expect the hugemem change to not put a cortex into an unstartable state. Bonus that there is documentation on how to recover from a state like this.
**Environment (please complete the following information):**
- Ubuntu 20.04.4 LTS
- vertexproject/synapse-cortex:master
- version from vertexproject/synapse-cortex:master
- Output of `python -m pip freeze` I can't launch the cortex in order to run this
**Additional context**
cortex_0_1 | {"message": "Updating hugenum index values: /vertex/storage/layers/9261972f63e1c41338fb60648d2b67fd", "logger": {"name": "synapse.lib.layer", "process": "MainProcess", "filename": "layer.py", "func": "_layrV7toV8"}, "level": "WARNING", "time": "2022-03-15 22:12:14,352"}
cortex_0_1 | {"message": "...complete!", "logger": {"name": "synapse.lib.layer", "process": "MainProcess", "filename": "layer.py", "func": "_layrV7toV8"}, "level": "WARNING", "time": "2022-03-15 22:12:14,355"}
cortex_0_1 | Traceback (most recent call last):
cortex_0_1 | File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
cortex_0_1 | return _run_code(code, main_globals, None,
cortex_0_1 | File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
cortex_0_1 | exec(code, run_globals)
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/servers/cortex.py", line 8, in <module>
cortex_0_1 | asyncio.run(s_cortex.Cortex.execmain(sys.argv[1:]))
cortex_0_1 | File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
cortex_0_1 | return loop.run_until_complete(main)
cortex_0_1 | File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
cortex_0_1 | return future.result()
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/lib/cell.py", line 2406, in execmain
cortex_0_1 | cell = await cls.initFromArgv(argv, outp=outp)
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/lib/cell.py", line 2353, in initFromArgv
cortex_0_1 | cell = await cls.anit(opts.dirn, conf=conf)
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/lib/base.py", line 97, in anit
cortex_0_1 | await self.__anit__(*args, **kwargs)
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/lib/cell.py", line 1026, in __anit__
cortex_0_1 | await self.initServiceRuntime()
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/cortex.py", line 1324, in initServiceRuntime
cortex_0_1 | await self._checkLayerModels()
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/cortex.py", line 3490, in _checkLayerModels
cortex_0_1 | await mrev.revCoreLayers()
cortex_0_1 | File "/usr/local/lib/python3.8/site-packages/synapse/lib/modelrev.py", line 374, in revCoreLayers
cortex_0_1 | raise s_exc.CantRevLayer(layer=layr.iden, mesg=mesg, curv=version, layv=vers)
cortex_0_1 | synapse.exc.CantRevLayer: CantRevLayer: curv=(0, 2, 7) layer='9261972f63e1c41338fb60648d2b67fd' layv=(0, 2, 8) mesg='layer Layer 9261972f63e1c41338fb60648d2b67fd (/vertex/storage/layers/9261972f63e1c41338fb60648d2b67fd) is from the future!'
ti-analysis-synapse_cortex_0_1 exited with code 1
Thank you
:id = 2597
:repo = dbfc7e21bd54adb324521c64dfaee15a
:title = [bug] updating hugenum index values results in cantrevlayer: layer is from the future error and container exit
:updated = 2022/03/17 16:40:53.000
:url = https://github.com/vertexproject/synapse/issues/2597
it:dev:repo:issue=2dcdc213acf72eaa1f4adead6ce67f43
.created = 2024/12/20 18:02:49.795
:created = 2022/02/26 19:31:14.000
:desc = **Describe the bug**
Currently, when you create a `inet:url` node, the values that get parsed out of the `:base` value for the `:user` and `:passwd` props are taken as literal values. However, for cases where a restricted character (e.g., `@`) is present in one of those values, it must be URL encoded to properly be sent to the server.
From my perspective, it makes sense to URL decode the `:user` and `:passwd` properties so that the resulting `inet:user` and `inet:passwd` nodes can be pivoted on more widely for cases where they may be observed without URL encoding.
**To Reproduce**
```
> [inet:url="smtp://myemail%40domain.com:p%[email protected]:587"]
inet:url=smtp://myemail%40domain.com:p%[email protected]:587
.created = 2022/02/26 19:28:14.569
:base = smtp://myemail%40domain.com:p%[email protected]:587
:fqdn = smtp.gmail.com
:params =
:passwd = p%40ssw0rd1
:path =
:port = 587
:proto = smtp
:user = myemail%40domain.com
```
The username and password properties still have the `@` character URL encoded, preventing further pivoting.
**Expected behavior**
```
> [inet:url="smtp://myemail%40domain.com:p%[email protected]:587"]
inet:url=smtp://myemail%40domain.com:p%[email protected]:587
.created = 2022/02/26 19:28:14.569
:base = smtp://myemail%40domain.com:p%[email protected]:587
:fqdn = smtp.gmail.com
:params =
:passwd = p@ssw0rd1
:path =
:port = 587
:proto = smtp
:user = [email protected]
```
The `@` in the username and password properties has been decoded properly, but is still URL encoded in the `:base` value since that's how it was literally observed.
**Environment (please complete the following information):**
- Synapse 2.82.1
- Optic 2.37.0
:id = 2568
:repo = dbfc7e21bd54adb324521c64dfaee15a
:title = [bug] url encoded username/password values in inet:url nodes aren't decoded when parsed to their respective properties
:updated = 2023/06/09 14:21:14.000
:url = https://github.com/vertexproject/synapse/issues/2568
Ingest only the open issues for the vivisect/vivisect repository, dynamically creating the parent it:dev:repo node:
> github.issues.pull --repo vivisect/vivisect --yield --size 5 --state open
WARNING: The property it:dev:repo:created is deprecated or using a deprecated type and will be removed in 3.0.0
WARNING: The property it:dev:repo:issue:created is deprecated or using a deprecated type and will be removed in 3.0.0
it:dev:repo:issue=d12ced02d49f556d20eb7c179f017603
.created = 2024/12/20 18:02:50.153
:created = 2023/07/03 18:54:17.000
:desc = https://github.com/vivisect/vivisect/blob/5655b84e23c6135e806fc67aba185dacefd771c7/setup.py#L35
https://pypi.org/project/cxxfilt/
:id = 623
:repo = ee35e32fad9696577f7d001170f24db0
:title = upgrade to cxxfilt 3.0
:updated = 2023/07/03 18:54:17.000
:url = https://github.com/vivisect/vivisect/issues/623
it:dev:repo:issue=fece6311b4ace8d29adfa0ac6cc530af
.created = 2024/12/20 18:02:50.240
:created = 2023/06/06 03:51:22.000
:desc = Hi, I was suggested to post this issue here (see https://github.com/mandiant/flare-floss/issues/743, https://github.com/mandiant/flare-floss/issues/778). I am encountering very high ram usage and extremely long processing times when analyzing a sample.
I tried to analyze LINE.exe (version: 7.17.0.3035, sha256: 37aa29719c130e7b86d84253bd5fefa1ebeeb1b516bf7ea0c250ffeaa932833b, virus total: https://www.virustotal.com/gui/file/37aa29719c130e7b86d84253bd5fefa1ebeeb1b516bf7ea0c250ffeaa932833b) I get after installing https://desktop.line-scdn.net/win/new/LineInst.exe. I know this exe is Themida protected, which might complicate things.
:id = 618
:repo = ee35e32fad9696577f7d001170f24db0
:title = performance issues
:updated = 2023/06/30 08:00:05.000
:url = https://github.com/vivisect/vivisect/issues/618
it:dev:repo:issue=ec35fa3c493b404026838c40842b7979
.created = 2024/12/20 18:02:50.326
:created = 2023/05/20 12:48:44.000
:desc = I am a vivisect newbie, so please forgive me should my request be trivial or already implemented.
Would it be possible to add a "step into" feature which runs as a continuous cycle, without having to select "step into" each time? I tried using "continuous mode" to analyze a piece of malware, but I only see the loading and unloading of the binaries and not the individual assembler commands, as you do when you use the "step into" option.
The issue is, I'm encountering some smart malware that must reference the system clock to see how long it takes for the code execution. Obviously, this exits the code prematurely.
If possible adding (or letting me know how to do it in VDB) a feature that traces the assembler commands in real-time, as well as recording the state of each of the registers, would be a really nice feature for vivisect.
So far, your program looks amazing! I just stumbled upon it today.
:id = 616
:repo = ee35e32fad9696577f7d001170f24db0
:title = vdb - stubborn malware using system time
:updated = 2023/05/25 10:28:24.000
:url = https://github.com/vivisect/vivisect/issues/616
it:dev:repo:issue=c15fb69e199d8248f305b35d7845c08a
.created = 2024/12/20 18:02:50.410
:created = 2023/04/13 13:39:34.000
:desc = from the CLI, `threads` returns the stack and PC values.
the GUI window includes a decimel number for "Thread Info" and an empty "State". these are meaningless fields at the moment. make valuable. include PC, if it's not too heavy a lift
:id = 612
:repo = ee35e32fad9696577f7d001170f24db0
:title = vdb threads window state bug
:updated = 2023/04/13 13:39:34.000
:url = https://github.com/vivisect/vivisect/issues/612
it:dev:repo:issue=5f6bc3c3981d5faecbb316e8e43bde1a
.created = 2024/12/20 18:02:50.496
:created = 2023/03/29 17:21:34.000
:desc = allow us to modify the function API (cconv, rettype, #args) from the gui
as an example, this is crappy little plugin i'm using right now:
for context, this plugin is the product of like 5 minutes of code, but has one remaining bug: if you reduce the number of args, the Function block doesn't render correctly. as soon as i spend another 20 minutes to figure that out, i should be able to carve it out of my toolbelt and make a PR.
:id = 602
:repo = ee35e32fad9696577f7d001170f24db0
:title = feature request: api modification
:updated = 2023/03/29 17:21:34.000
:url = https://github.com/vivisect/vivisect/issues/602
Ingest every issue for the vivisect/vivisect repository:
> it:dev:repo:name=vivisect/vivisect | github.issues.pull --yield --size 5
it:dev:repo:issue=d12ced02d49f556d20eb7c179f017603
.created = 2024/12/20 18:02:50.153
:created = 2023/07/03 18:54:17.000
:desc = https://github.com/vivisect/vivisect/blob/5655b84e23c6135e806fc67aba185dacefd771c7/setup.py#L35
https://pypi.org/project/cxxfilt/
:id = 623
:repo = ee35e32fad9696577f7d001170f24db0
:title = upgrade to cxxfilt 3.0
:updated = 2023/07/03 18:54:17.000
:url = https://github.com/vivisect/vivisect/issues/623
it:dev:repo:issue=fece6311b4ace8d29adfa0ac6cc530af
.created = 2024/12/20 18:02:50.240
:created = 2023/06/06 03:51:22.000
:desc = Hi, I was suggested to post this issue here (see https://github.com/mandiant/flare-floss/issues/743, https://github.com/mandiant/flare-floss/issues/778). I am encountering very high ram usage and extremely long processing times when analyzing a sample.
I tried to analyze LINE.exe (version: 7.17.0.3035, sha256: 37aa29719c130e7b86d84253bd5fefa1ebeeb1b516bf7ea0c250ffeaa932833b, virus total: https://www.virustotal.com/gui/file/37aa29719c130e7b86d84253bd5fefa1ebeeb1b516bf7ea0c250ffeaa932833b) I get after installing https://desktop.line-scdn.net/win/new/LineInst.exe. I know this exe is Themida protected, which might complicate things.
:id = 618
:repo = ee35e32fad9696577f7d001170f24db0
:title = performance issues
:updated = 2023/06/30 08:00:05.000
:url = https://github.com/vivisect/vivisect/issues/618
it:dev:repo:issue=ec35fa3c493b404026838c40842b7979
.created = 2024/12/20 18:02:50.326
:created = 2023/05/20 12:48:44.000
:desc = I am a vivisect newbie, so please forgive me should my request be trivial or already implemented.
Would it be possible to add a "step into" feature which runs as a continuous cycle, without having to select "step into" each time? I tried using "continuous mode" to analyze a piece of malware, but I only see the loading and unloading of the binaries and not the individual assembler commands, as you do when you use the "step into" option.
The issue is, I'm encountering some smart malware that must reference the system clock to see how long it takes for the code execution. Obviously, this exits the code prematurely.
If possible adding (or letting me know how to do it in VDB) a feature that traces the assembler commands in real-time, as well as recording the state of each of the registers, would be a really nice feature for vivisect.
So far, your program looks amazing! I just stumbled upon it today.
:id = 616
:repo = ee35e32fad9696577f7d001170f24db0
:title = vdb - stubborn malware using system time
:updated = 2023/05/25 10:28:24.000
:url = https://github.com/vivisect/vivisect/issues/616
it:dev:repo:issue=c15fb69e199d8248f305b35d7845c08a
.created = 2024/12/20 18:02:50.410
:created = 2023/04/13 13:39:34.000
:desc = from the CLI, `threads` returns the stack and PC values.
the GUI window includes a decimel number for "Thread Info" and an empty "State". these are meaningless fields at the moment. make valuable. include PC, if it's not too heavy a lift
:id = 612
:repo = ee35e32fad9696577f7d001170f24db0
:title = vdb threads window state bug
:updated = 2023/04/13 13:39:34.000
:url = https://github.com/vivisect/vivisect/issues/612
WARNING: The property it:dev:repo:issue:created is deprecated or using a deprecated type and will be removed in 3.0.0
it:dev:repo:issue=e967fa213431642335dccbe3d4a1b466
.created = 2024/12/20 18:02:51.066
:created = 2023/04/13 13:33:13.000
:desc = not sure when yet, but Interactive Python window has become unusable.
CLI python still works, but when i attempt to run code from the Interactive Python window, I get this error:
:id = 611
:repo = ee35e32fad9696577f7d001170f24db0
:title = interactive python bug
:updated = 2023/05/07 23:36:11.000
:url = https://github.com/vivisect/vivisect/issues/611
View the labels an issue has been marked with:
> it:dev:repo:issue | limit 1 | -> it:dev:repo:issue:label -> it:dev:repo:label
it:dev:repo:label=16cc593845f4d9d7bce8f610a817adb6
.created = 2024/12/20 18:02:49.085
:id = 645406424
:title = needs triage
Ingest the comments for a GitHub issue
Ingest all the comments for a particular GitHub repository:
> it:dev:repo:name=vivisect/vivisect -> it:dev:repo:issue | github.issues.comments --yield --size 5
WARNING: The property it:dev:repo:issue:comment:created is deprecated or using a deprecated type and will be removed in 3.0.0
it:dev:repo:issue:comment=e5299d0584425c6e592f43de674ae595
.created = 2024/12/20 18:02:51.264
:created = 2023/06/06 07:42:25.000
:issue = fece6311b4ace8d29adfa0ac6cc530af
:text = Thanks, @Dobatymo for reporting this here. As additional context: IDA only runs a few seconds on the sample, so something appears to be off (when discovering code?).
A similar sample (also Themida packed) is [9cc387fd485e91fc58a626d2c64b85e0502ba60f3718afd7b5fd6e5b46721bb9](https://www.virustotal.com/gui/file/9cc387fd485e91fc58a626d2c64b85e0502ba60f3718afd7b5fd6e5b46721bb9).
:updated = 2023/06/06 07:42:25.000
:url = https://github.com/vivisect/vivisect/issues/618#issuecomment-1578101783
it:dev:repo:issue:comment=29fe81f361523520db97d240ee0b70db
.created = 2024/12/20 18:02:51.285
:created = 2023/06/28 20:49:20.000
:issue = fece6311b4ace8d29adfa0ac6cc530af
:replyto = e5299d0584425c6e592f43de674ae595
:text = sorry, this has been a crazy few weeks, and i'm just getting over being sick, but i wanted to check on this. how good an analysis does IDA do? they don't do emulation by default. is there something else that they may not be doing?
i'm not trying to cast stones and i don't want to blow you off. i want to get a good representation of ground truth so we can emulate apples to apples... ultimately allowing us to know when we are done.
thanks!
@
:updated = 2023/06/28 20:49:20.000
:url = https://github.com/vivisect/vivisect/issues/618#issuecomment-1612086763
it:dev:repo:issue:comment=13ebca80b66aa858f26f10629dea9c3d
.created = 2024/12/20 18:02:51.305
:created = 2023/06/29 01:38:20.000
:issue = fece6311b4ace8d29adfa0ac6cc530af
:replyto = 29fe81f361523520db97d240ee0b70db
:text = continuing to look into this, but here's probably the problem:
That 30MB function is simply not what Viv expects. lol.
we'll check into what's really causing the delay, but it could be purely two things, neither of which are likely to change:
* Vivisect is Pure Python - it brings great benefits for making it easy to understand, extend, and write analysis tools for... but it's going to be slower than C/C++ tools
* Vivisect annotates *every* location in a database. that takes time and memory to pre-calculate, but it makes xrefs and location finding super simple and cost-effective when looking up what lives at a given location and how big it is. to ensure that i'm being clear, *every* asm opcode gets its own location entry. i'm pretty sure IDA does block level tracking, which can cause it to get confused more easily when analyzing code that doesn't want to be analyzed.
still, we'll see if we can make this more performant...
it could be something completely different as well.
@rakuy0 : (for your awareness)
:updated = 2023/06/29 01:38:20.000
:url = https://github.com/vivisect/vivisect/issues/618#issuecomment-1612309824
it:dev:repo:issue:comment=03fa7a7b5567d697577cb290c4789f8c
.created = 2024/12/20 18:02:51.324
:created = 2023/06/30 08:00:05.000
:issue = fece6311b4ace8d29adfa0ac6cc530af
:replyto = 13ebca80b66aa858f26f10629dea9c3d
:text = Thanks for taking a look. I've referenced IDA because I' suspected that viv's code analysis goes off the rails.
In IDA (9cc387fd485e91fc58a626d2c64b85e0502ba60f3718afd7b5fd6e5b46721bb9:0x142B10E80 isn't big at all):
<img width="454" alt="2023-06-30_09-51-06_ida64" src="https://github.com/vivisect/vivisect/assets/17606537/a1e9ca6f-f773-4793-b380-05b8b8b3d2eb">
Just guessing, but could these weird jmps/calls cause this?
<img width="1093" alt="2023-06-30_09-56-59_ida64" src="https://github.com/vivisect/vivisect/assets/17606537/87a0003d-59b9-4e6a-9c5c-81b5bf21e822">
:updated = 2023/06/30 08:00:05.000
:url = https://github.com/vivisect/vivisect/issues/618#issuecomment-1614281077
it:dev:repo:issue:comment=233117de9e6d8eb3cfde78ec75d637d6
.created = 2024/12/20 18:02:51.405
:created = 2023/05/22 16:13:28.000
:issue = ec35fa3c493b404026838c40842b7979
:text = hey @99bobster99 ! good thinking.
so.... there is no easy one-shot way to do what you're asking.
we *do* support iterative Stepi (see `stepi` help from the command prompt), even tracking what the arguments are (if you're willing to delay execution even more, see `stepi -V`)
a few of the options i use pretty regularly:
```
stepi -A <address>
stepi -B
stepi -C <number of instructions to stepi>
stepi -R
stepi -U <until clause>
```
the only way to stepi "in real time" is to set a breakpoint and resume execution. that doesn't give you assembly instructions either. but it may defeat the timer-based testing.
another method i've used:
* stepi -A 0 # basically, stepi until death - best used with the `saveout` command
* look through the instructions for things like calls to timer functions (or specifically the `rdtsc` ASM instruction, typically pairs)
* rerun with a breakpoint after that instruction
* wash, rinse, repeat
or something similar.
@rakuy0 this is an example where we need #608 merged in to enable better `saveout` functionality :)
hope this helps!
@
:updated = 2023/05/22 16:13:28.000
:url = https://github.com/vivisect/vivisect/issues/616#issuecomment-1557516307
it:dev:repo:issue:comment=d1207e6750bb8423777cc4807e6ea570
.created = 2024/12/20 18:02:51.424
:created = 2023/05/22 20:53:48.000
:issue = ec35fa3c493b404026838c40842b7979
:replyto = 233117de9e6d8eb3cfde78ec75d637d6
:text = Hey atlas0fd00, thank you for the detailed response!
I'm getting some results now, using "Stepi -A 0 #". :) Is it normal for the process to stop responding, while the "Stepi -A 0 #" command is running and logging?
Quick question, the memory canvas doesn't seem to be working for me, is this a known issue?
```
AttributeError("'CanvasMethodProxy' object has no attribute 'getSymByAddr'")
AttributeError("'CanvasMethodProxy' object has no attribute 'getSymHint'")
```
I'd like to learn more about ViviSect, but I'm not finding a lot of Vivisect tutorials on the web.
I found this [one](https://github.com/vivisect/vivisect/issues/120), that directs you to a (3) part series done over at INAP, but it looks like only "part 1" is still available. Most of the references on this old 2016 vivisect issue response are dead. :(
Do you happen to have a copy of the other (2) INAP parts, or are there any other tutorials you'd recommend?
[INAP - Part 2](https://www.singlehop.com/blog/binary-vivisection-part-2/)
[INAP - Part 3](https://www.singlehop.com/blog/binary-vivisection-part-3/)
:updated = 2023/05/23 13:24:55.000
:url = https://github.com/vivisect/vivisect/issues/616#issuecomment-1557976673
it:dev:repo:issue:comment=dca3caee0d96d3e8f6d94bf6e5165e57
.created = 2024/12/20 18:02:51.444
:created = 2023/05/24 16:48:08.000
:issue = ec35fa3c493b404026838c40842b7979
:replyto = d1207e6750bb8423777cc4807e6ea570
:text = hey @99bobster99 ,
yes, it's common for the UI to stop responding, and the process as well, while single-stepping. single-stepping it wildly inefficient, requiring the OS to get involved and return control back to VDB after each instruction. using the non-GUI mode for this is at least somewhat better, since the GUI can have a bunch of windows that think they deserve to get updated after each stepi.
and please accept my apologies for the ugliness. we've discovered a number of bugs in the VDB code that we have recently (or are working on) fixed but hasn't made it into a release.
i've been working on a list of tutorial things to make Vivisect more accessible. i'm not sure how much will be Video or otherwise, but i'm interested in help. care to help out? it'd be great experience, and we might just make Vivisect/VDB better suit your needs in the process :)
currently the only docs i can point you to are at https://vivisect.readthedocs.io
@
:updated = 2023/05/24 16:48:08.000
:url = https://github.com/vivisect/vivisect/issues/616#issuecomment-1561579285
it:dev:repo:issue:comment=5a137b398afc79a68b61beb054b259b6
.created = 2024/12/20 18:02:51.463
:created = 2023/05/24 22:04:43.000
:issue = ec35fa3c493b404026838c40842b7979
:replyto = dca3caee0d96d3e8f6d94bf6e5165e57
:text = wow, i just found this today. it's not really about VDB, per se, but is a good intro to Vivisect itself:
https://www.youtube.com/watch?v=K55b8w39oC8
:updated = 2023/05/24 22:04:43.000
:url = https://github.com/vivisect/vivisect/issues/616#issuecomment-1561985407
it:dev:repo:issue:comment=9f1c0f36491c1097688711f89108ee4e
.created = 2024/12/20 18:02:51.484
:created = 2023/05/25 10:28:24.000
:issue = ec35fa3c493b404026838c40842b7979
:replyto = 5a137b398afc79a68b61beb054b259b6
:text = Hey atlas0fd00,
No need to apologize, Vivisect looks awesome already! Absolutely, I'd love to help out, with whatever I can contribute. Send me any newer version you'd like me to test drive. I have a few other binaries I'm playing around with in order to learn more about reversing.
I've also been playing around with IDA Pro, x64dbg and Immunity Debugger. So far I like the "environment" isolation approach the best, that you've structured Vivisect around.
Thank you for the Youtube link to Vivisect/Vivida, I'm watching it now! :)
:updated = 2023/05/25 10:28:24.000
:url = https://github.com/vivisect/vivisect/issues/616#issuecomment-1562665320
it:dev:repo:issue:comment=900cc6d651e7ae4625104ef6b42efb92
.created = 2024/12/20 18:02:51.616
:created = 2023/05/07 23:36:11.000
:issue = e967fa213431642335dccbe3d4a1b466
:text = fixed #613
:updated = 2023/05/07 23:36:11.000
:url = https://github.com/vivisect/vivisect/issues/611#issuecomment-1537568328
Use of meta:source nodes
Synapse-GitHub uses a meta:source node and -(seen)> light
weight edges to track nodes observed from the GitHub API.
> meta:source=fcd253a79765fe20d7a5a8e1dd0a681d
meta:source=fcd253a79765fe20d7a5a8e1dd0a681d
.created = 2024/12/20 18:02:48.322
:name = github api
Storm can be used to filter nodes to include/exclude nodes which have been observed by Synapse-GitHub. The following example shows how to filter the results of a query to include only results observed by Synapse-GitHub:
> #cool.tag.lift +{ <(seen)- meta:source=fcd253a79765fe20d7a5a8e1dd0a681d }