Commit Graph

2755 Commits (d1cdc3efe3cf4057062eb86f446b510c17e72c6e)

Author SHA1 Message Date
Zlatin Balevsky d00d6d7e18
flatten the nameToFile map as well 2021-06-05 23:39:25 +01:00
Zlatin Balevsky 71d20d075b
reduce memory usage of the rootToFiles structure by switching to array values 2021-06-05 23:18:39 +01:00
Zlatin Balevsky 30abc5af8d
get rid of unused cached value 2021-06-05 22:16:49 +01:00
Zlatin Balevsky fe85887b39
use two SkipLists, thanks zzz for the suggestion 2021-06-05 15:44:53 +01:00
Zlatin Balevsky 72c4998954
increment counters appropriately after increasing value size to integer 2021-06-05 05:29:54 +01:00
Zlatin Balevsky ea1f431323
more efficient splitting, reduces blockfile access 2021-06-05 03:18:18 +01:00
Zlatin Balevsky cb2ab77316
increase max value size to 2^24 2021-06-05 03:11:50 +01:00
Zlatin Balevsky a9c4f612fd
fix re-use of SkipSpan when it goes empty 2021-06-05 03:04:52 +01:00
Zlatin Balevsky 1168a871eb
get rid of the limit for byte[] i/o 2021-06-04 22:27:22 +01:00
Zlatin Balevsky 85043abb82 Merge branch 'blockfile-index' into 'master'
Off-heap Blockfile storage for search index

See merge request zlatinb/muwire!52
2021-06-04 17:49:52 +00:00
Zlatin Balevsky 0916e67939
rename package so that it won't conflict in the plugin 2021-06-04 18:22:45 +01:00
Zlatin Balevsky 88f1f381b9
multi-part backing storage 2021-06-04 18:14:07 +01:00
Zlatin Balevsky 4a0275914d
clean or suppress some warnings 2021-06-04 16:05:14 +01:00
Zlatin Balevsky 38bfecbc4f
increase span size 2021-06-04 16:01:14 +01:00
Zlatin Balevsky 40492ac5d2
re-add nulling of values 2021-06-04 16:00:31 +01:00
Zlatin Balevsky 008d82ad1d
side-port the blockfile db changes from i2p.i2p 2021-06-04 14:54:16 +01:00
Zlatin Balevsky 93ac56b353
fix or suppress various warnings 2021-06-04 12:41:01 +01:00
Zlatin Balevsky 84387bef8e
clear out vals storage after retrieving value 2021-06-04 12:35:21 +01:00
Zlatin Balevsky 3d30f49776
do not store values in the heap, just keys 2021-06-04 12:23:22 +01:00
Zlatin Balevsky 9ecb73149c
Split processing of files from directories. Throttle emission of new events based on threadpool queue length 2021-06-04 05:57:50 +01:00
Zlatin Balevsky dfd7be9618
reduce the size of the memmapped file 2021-06-04 03:48:15 +01:00
Zlatin Balevsky 8892dd73a8
switch to a memory-mapped file instead of a RAF 2021-06-04 03:21:43 +01:00
Zlatin Balevsky 76eaca996f
experimental Blockfile db instead of in-memory index 2021-06-04 00:09:33 +01:00
Zlatin Balevsky 1c2972ecfc
add some default file types to ignore 2021-06-03 17:22:38 +01:00
Zlatin Balevsky 34b7496036
prevent corrupt messages from breaking messenger loading 2021-06-03 12:54:00 +01:00
Zlatin Balevsky 6f3a3357ea
more efficient traversal of directories 2021-06-03 12:15:14 +01:00
Zlatin Balevsky 3ac6d085f6
configurable list of file types to not share 2021-06-03 05:27:42 +01:00
Zlatin Balevsky 95fcd1635f
more safeguards against zero-length or unreadable files 2021-06-03 01:59:33 +01:00
Zlatin Balevsky 4b11aca16e
show the number of files being hashed 2021-06-02 20:10:26 +01:00
Zlatin Balevsky 2af996266f
check if file is readable before trying to hash 2021-06-02 20:09:55 +01:00
Zlatin Balevsky c849fa078a
No need to restart when chaning the number of hashing threads , GitHub issue #57 2021-06-02 19:17:17 +01:00
Zlatin Balevsky b1af2e557b
Make the number of hashing threads configurable, GitHub issue #57 2021-06-01 23:35:59 +01:00
Zlatin Balevsky 228a92a58d
filter the hosts that are about to be verified as well 2021-06-01 20:24:38 +01:00
Zlatin Balevsky 0fc5ca7141
fix copy-full-id from message table 2021-06-01 19:25:29 +01:00
Zlatin Balevsky 47e2c32877
update TODO with some items and ideas 2021-06-01 19:09:19 +01:00
Zlatin Balevsky 173fd55afb
update version, get rid of cli completely 2021-05-26 22:34:36 +01:00
Zlatin Balevsky 7a40cdd0c2
return 3 hosts from hostcache 2021-05-26 09:42:41 +01:00
Zlatin Balevsky 837b8d8213
connect more aggressively, send more pongs, fix destination parsing and header check 2021-05-26 08:57:20 +01:00
Zlatin Balevsky 4e09b4ad71
forgot 0x 2021-05-26 08:01:48 +01:00
Zlatin Balevsky cfe1fe98b0
wip on switching to binary pongs 2021-05-26 07:46:25 +01:00
Zlatin Balevsky 8f1b663bbd
document binary pongs 2021-05-26 06:57:55 +01:00
Zlatin Balevsky 51f6a49459
translate retry/resume popup text 2021-05-25 23:50:04 +01:00
Zlatin Balevsky 99e9eb00a1
reduce the request q size if the file has few pieces 2021-05-24 11:53:29 +01:00
Zlatin Balevsky 4cff64133e
correct space check 2021-05-24 11:52:15 +01:00
Zlatin Balevsky a584f16ce4
properly terminate headers 2021-05-24 11:50:50 +01:00
Zlatin Balevsky ce787a6f99
avoid BigIntegers in progress bar rendering 2021-05-24 11:49:27 +01:00
Zlatin Balevsky 0e658faee8
forgot terminating rn 2021-05-24 05:06:51 +01:00
Zlatin Balevsky 5ded9c4f30
fix collections warning always showing even if there are no collections 2021-05-24 04:28:01 +01:00
Zlatin Balevsky f9ceb7854a
use KiB for download progress as well 2021-05-24 03:56:27 +01:00
Zlatin Balevsky d243e1cd31
do not add 1 to the number of pieces, let the HEAD request tell the value 2021-05-24 03:52:31 +01:00