Can not compare a file and same file stored in mysql - sha256
-
I'm storing a file in a mysql table in a Longblob. Now i would like to create a sha256 as a "footprint" for this file, so i can verify if the file exists already in my database. As a test i've created this code
private byte\[\] ObjectToByteArray(Object obj) { if (obj == null) return null; BinaryFormatter bf = new BinaryFormatter(); MemoryStream ms = new MemoryStream(); bf.Serialize(ms, obj); return ms.ToArray(); } private void button5\_Click(object sender, EventArgs e) { byte\[\] bookData = null; byte\[\] bookDataSha = null; book bog = getBookById("6"); bog.GetEbookHash(string.Format(@"H:\\calibre\\{0}\\", bog.path), 6); using (var db = new MySqlConnection(@"server = xxxxxxx")) { MySql.Data.MySqlClient.MySqlDataReader myData; db.Open(); MySqlCommand comm = db.CreateCommand(); comm.CommandText = "select content from bookContent where id='6' and type='epub'"; myData = comm.ExecuteReader(); while (myData.Read()) { bookData = this.ObjectToByteArray(myData\["content"\]); bookDataSha = Sha256.ComputeHash(bookData); } db.Close(); } if (bog.EpubHash == bookDataSha) { MessageBox.Show("success"); } MessageBox.Show(bog.getEbookHashStr(bog.EpubHash)+" - "+bog.getEbookHashStr(bookDataSha)); }My problem is that the 2 sha256's isnt equal. I've found this Generate a SHA-256 encrypted hash[^] and when i upload the file i test on, it creates the same sha256 as my code for the file does. But when i read the data from the longblob and creates a sha256 it is different. The strange part, is that if i use my website to download the same longblob and saves it as a file... FC cant find any diffeneces. What do i do wrong ? Thanks in advance
Yours Wilco
-
I'm storing a file in a mysql table in a Longblob. Now i would like to create a sha256 as a "footprint" for this file, so i can verify if the file exists already in my database. As a test i've created this code
private byte\[\] ObjectToByteArray(Object obj) { if (obj == null) return null; BinaryFormatter bf = new BinaryFormatter(); MemoryStream ms = new MemoryStream(); bf.Serialize(ms, obj); return ms.ToArray(); } private void button5\_Click(object sender, EventArgs e) { byte\[\] bookData = null; byte\[\] bookDataSha = null; book bog = getBookById("6"); bog.GetEbookHash(string.Format(@"H:\\calibre\\{0}\\", bog.path), 6); using (var db = new MySqlConnection(@"server = xxxxxxx")) { MySql.Data.MySqlClient.MySqlDataReader myData; db.Open(); MySqlCommand comm = db.CreateCommand(); comm.CommandText = "select content from bookContent where id='6' and type='epub'"; myData = comm.ExecuteReader(); while (myData.Read()) { bookData = this.ObjectToByteArray(myData\["content"\]); bookDataSha = Sha256.ComputeHash(bookData); } db.Close(); } if (bog.EpubHash == bookDataSha) { MessageBox.Show("success"); } MessageBox.Show(bog.getEbookHashStr(bog.EpubHash)+" - "+bog.getEbookHashStr(bookDataSha)); }My problem is that the 2 sha256's isnt equal. I've found this Generate a SHA-256 encrypted hash[^] and when i upload the file i test on, it creates the same sha256 as my code for the file does. But when i read the data from the longblob and creates a sha256 it is different. The strange part, is that if i use my website to download the same longblob and saves it as a file... FC cant find any diffeneces. What do i do wrong ? Thanks in advance
Yours Wilco
Hi, I'm not familiar with the MySql connector, and I tend not to store files in databases (it does not make much sense to me, it only inflates the DB size). However I noted what follows: at the bottom of your code there is a while loop, wherein you call
Sha256.ComputeHash. That piece of code looks very suspicious: what remains is just the SHA256 of the last loop iteration, so either you only have one iteration (and the while should be an if), or your logic is wrong (you should collect all the data and then calculate SHA256 on the whole thing). You could try with a small file, say less than 1KB. Or you could track the while loop behavior, either by adding an output statement, or by using a debugger... :)Luc Pattyn [My Articles] Nil Volentibus Arduum
-
I'm storing a file in a mysql table in a Longblob. Now i would like to create a sha256 as a "footprint" for this file, so i can verify if the file exists already in my database. As a test i've created this code
private byte\[\] ObjectToByteArray(Object obj) { if (obj == null) return null; BinaryFormatter bf = new BinaryFormatter(); MemoryStream ms = new MemoryStream(); bf.Serialize(ms, obj); return ms.ToArray(); } private void button5\_Click(object sender, EventArgs e) { byte\[\] bookData = null; byte\[\] bookDataSha = null; book bog = getBookById("6"); bog.GetEbookHash(string.Format(@"H:\\calibre\\{0}\\", bog.path), 6); using (var db = new MySqlConnection(@"server = xxxxxxx")) { MySql.Data.MySqlClient.MySqlDataReader myData; db.Open(); MySqlCommand comm = db.CreateCommand(); comm.CommandText = "select content from bookContent where id='6' and type='epub'"; myData = comm.ExecuteReader(); while (myData.Read()) { bookData = this.ObjectToByteArray(myData\["content"\]); bookDataSha = Sha256.ComputeHash(bookData); } db.Close(); } if (bog.EpubHash == bookDataSha) { MessageBox.Show("success"); } MessageBox.Show(bog.getEbookHashStr(bog.EpubHash)+" - "+bog.getEbookHashStr(bookDataSha)); }My problem is that the 2 sha256's isnt equal. I've found this Generate a SHA-256 encrypted hash[^] and when i upload the file i test on, it creates the same sha256 as my code for the file does. But when i read the data from the longblob and creates a sha256 it is different. The strange part, is that if i use my website to download the same longblob and saves it as a file... FC cant find any diffeneces. What do i do wrong ? Thanks in advance
Yours Wilco
The usual method for this is to store the file and the hash in the database, then you only need to compare the hash rather than recompute each time. The better method is to store the file in the file system and only store the filename and path with the hash in the database.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
-
The usual method for this is to store the file and the hash in the database, then you only need to compare the hash rather than recompute each time. The better method is to store the file in the file system and only store the filename and path with the hash in the database.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
Hi Mycroft That is the plan, when i come there... but first step is to be able to compare the 2 hash values. If i upload a file to the database with a hash, it's ok, but for database maintenance purpose i would like to be able to recalculate the value "on the fly", with out having to save all 3000+ files one at a time to recalculate hash value. And i just can wrap my brain around the difference... basicly its an array of bytes - both the longblob and the filestream. As for saving the file to database... this a give premis for this task. I have very limited space in the OS environment, but unlimited space in the database.
Yours Wilco
-
Hi, I'm not familiar with the MySql connector, and I tend not to store files in databases (it does not make much sense to me, it only inflates the DB size). However I noted what follows: at the bottom of your code there is a while loop, wherein you call
Sha256.ComputeHash. That piece of code looks very suspicious: what remains is just the SHA256 of the last loop iteration, so either you only have one iteration (and the while should be an if), or your logic is wrong (you should collect all the data and then calculate SHA256 on the whole thing). You could try with a small file, say less than 1KB. Or you could track the while loop behavior, either by adding an output statement, or by using a debugger... :)Luc Pattyn [My Articles] Nil Volentibus Arduum
Hi Luc You are abosolute right. This example was copied from some where else i my code... I've changed it to a IF, but change in the hash value. I thought i would run this by the community to see if anybody have had a problem like this before.... I'll just lay it to side, and later look into the byte array differences... there has to be some, since the sha256 values are different.. Thanks for your response
Yours Wilco
-
I'm storing a file in a mysql table in a Longblob. Now i would like to create a sha256 as a "footprint" for this file, so i can verify if the file exists already in my database. As a test i've created this code
private byte\[\] ObjectToByteArray(Object obj) { if (obj == null) return null; BinaryFormatter bf = new BinaryFormatter(); MemoryStream ms = new MemoryStream(); bf.Serialize(ms, obj); return ms.ToArray(); } private void button5\_Click(object sender, EventArgs e) { byte\[\] bookData = null; byte\[\] bookDataSha = null; book bog = getBookById("6"); bog.GetEbookHash(string.Format(@"H:\\calibre\\{0}\\", bog.path), 6); using (var db = new MySqlConnection(@"server = xxxxxxx")) { MySql.Data.MySqlClient.MySqlDataReader myData; db.Open(); MySqlCommand comm = db.CreateCommand(); comm.CommandText = "select content from bookContent where id='6' and type='epub'"; myData = comm.ExecuteReader(); while (myData.Read()) { bookData = this.ObjectToByteArray(myData\["content"\]); bookDataSha = Sha256.ComputeHash(bookData); } db.Close(); } if (bog.EpubHash == bookDataSha) { MessageBox.Show("success"); } MessageBox.Show(bog.getEbookHashStr(bog.EpubHash)+" - "+bog.getEbookHashStr(bookDataSha)); }My problem is that the 2 sha256's isnt equal. I've found this Generate a SHA-256 encrypted hash[^] and when i upload the file i test on, it creates the same sha256 as my code for the file does. But when i read the data from the longblob and creates a sha256 it is different. The strange part, is that if i use my website to download the same longblob and saves it as a file... FC cant find any diffeneces. What do i do wrong ? Thanks in advance
Yours Wilco
Check the length of the blob that you are saving, vs. the length of the one you are reading. If you suspect the sha256-hash, then replace it with another hashing algo. As for "not storing files in a db" in support of exposing the filesystem; I would advise against it. Modern databases are fast enough to work as a file-system, and are used as such.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.