feature: Added (generate_libtoken_dict.sh) script for simplified work with libtokencap
#2106
Conversation
|
Hi, I do not mind merging this, however |
|
Yes, great, would be happy to share the scripts used. if you are happy with this example, I will add it to the README on the next commit. |
utils/libtokencap/make_dict_v2.sh
Outdated
| } >"$AFL_TOKEN_FILE" | ||
|
|
||
| # sort & remove duplicates | ||
| sort -u "$AFL_TOKEN_FILE" >"$AFL_DICT_FILE" |
There was a problem hiding this comment.
a) you still need to delete the tempfile and b) an echo that prints the generated filename would be good, right?
There was a problem hiding this comment.
I see the additional sorting by the system utility as justified, since we have no guarantee that the user will perform tokenization on the minimized dictionary. Therefore, i think there are two approaches here:
a)modify the libtokencap tool itself - cache values beforehand and then select unique values afterwards ;
b)use a unix-way style aid band for this task
There was a problem hiding this comment.
No, it makes no sense to keep AFL_TOKEN_FILE. AFL_DICT FILE is the same just deduplicated. so keeping AFL_TOKEN_FILE is pointless.
and you need to inform the user what the filename is that you generated. like echo Done! captured tokens written to ... etc.
utils/libtokencap/make_dict_v2.sh
Outdated
| { | ||
| touch "$AFL_TOKEN_FILE" | ||
| for i in $(find "$target_output" -type f -name "id*"); do | ||
| LD_PRELOAD="$LD_PRELOAD_PATH" \ |
There was a problem hiding this comment.
you should check first if LD_PRELOAD is unset before doing this
There was a problem hiding this comment.
okay, its been added to check flags.
utils/libtokencap/make_dict_v2.sh
Outdated
| @@ -0,0 +1,49 @@ | |||
| #default values | |||
| timeout_sec=5 | |||
| LD_PRELOAD_PATH="/home/${USER}/AFLplusplus/utils/libtokencap/libtokencap.so" | |||
There was a problem hiding this comment.
Such a default is a bad idea IHMO, just remove this line
utils/libtokencap/make_dict_v2.sh
Outdated
| echo "Error: Missing mandatory opts" >&2 | ||
| usage | ||
| fi | ||
|
|
There was a problem hiding this comment.
put a check here that error if both LD_PRELOAD and LD_PRELOAD_PATH are empty.
There was a problem hiding this comment.
seems yes, its a localized point, worth making more universal.
There was a problem hiding this comment.
that is not what I meant but thinking about it, what you did is better
|
just a few nits and then it is fine IMHO |
(-) removed default vars ; (+) added LD_PRELOAD_PATH check
utils/libtokencap/make_dict_v2.sh
Outdated
|
|
||
| # initialize vars | ||
| AFL_TOKEN_FILE="${PWD}/temp_output.txt" | ||
| AFL_DICT_FILE="$(basename "$target_output").dict" |
There was a problem hiding this comment.
I am not a fan of the AFL_DICT_FILE naming.
either use a fixed name (e.g. tokens.dict), and/or set it e.g. with an -f parameter, or base it on target_bin.
an output directory is usually out, o, corpus or corpus, so the generated filename is not helpful IMHO.
|
I would recommend to rename the tool. |
|
sorry this took 3 loops, I finally had time to really look at the script, I am drowning in work and various tasks. |
sigdevel
changed the title
libtokencaplibtokencap
Yes, that comment seems logical - I renamed in the dependent files :) |
That's okay If the tweaks helped the project a little bit - I was pleased. |
|
looks good, but still the temporary file is not deleted and the naming of the result file is not helpful + no final echo that prints the final filename plus that everything went fine. another thought: sometimes a target needs command line parameters. anything after '--' (getopt handles this) should be given to the target command ( |
Hi, based on your review, I've added changes to the file. |
|
LGTM thank you |
Prepared a script (make_dict.sh) for simplified work with libtokencap. The description of the work libtokencap is given in the file README.md , but to simplify the startup, it will probably be more convenient to add a separate script that can be used at once