den4b Forum

therube

Member

Registered: 2010-05-24

Posts: 31

RFE: Investigate New Hash Methods

RFE: Investigate New (non-cryptographic) Hash algorithms, be it xxHash or some other.

Was thinking more along the lines of (local) duplicate file checking (shift or control open opening/adding to Hasher), where xxHash (or other) should be faster compared to MD5/SHA1...

https://cyan4973.github.io/xxHash/
https://www.voidtools.com/forum/viewtop … f=4&t=8192

den4b

Administrator

From: den4b.com

Registered: 2006-04-06

Posts: 3,440

Re: RFE: Investigate New Hash Methods

It is possible to add xxHash to the list of supported hashing algorithms.

However, the benefit of this algorithm appears to be quite an edge case where the content to be hashed is being read from a storage at a speed greater than 5 GB/s, which is far above the limitations of the read speeds of common consumer file storage devices.

Conventional spinning disks can reach 200 MB/s, SATA based SSD drives can reach 500 MB/s, and even the top of the range Samsung NVMe SSD 970 PRO can barely reach 3GB/s in practice.

Do you have a scenario or a setup where you can extract value out of using xxHash algorithm?

---

A story of software development, and how it all evolves into Live vs Pro
Request a Wiki account | Reporting spam and abuse | Why did I get banned?