Converting List<int> to List<object> for One to Many relation with sqlite-net-extensions
-
Im creating a Xamarin.Android app and for database im using sqlite-net. I was trying to create a table with this structure
public class Templates
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public int Category { get; set; }
[TextBlob("imagesBlobbed")]
public List Images { get; set; }
public string imagesBlobbed { get; set; }
}
but i want every time im adding a template to check if it already exist by checking the Images like this:public static void AddTemplate(SQLiteConnection db, int category, List images)
{
var alreadyExist = db.Table().Where(record => record.Images == images);if (alreadyExist?.Count() == 0) { var template = new Templates() { Category = category, Images = images, }; db.InsertWithChildren(template, recursive: true); }}
but it's not working and i'm guessing it is not working because you cannot query with parameter "Images", in the database is stored as a blob. So im guessing i have to convert List to something else that is more easily supported in sqlite.I'm thinking of using sqlite-net-extensions that support one to many relations and do something like this:
[Table("Templates")]
public class Template
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public int Category { get; set; }
[OneToMany]
public TemplateImage Images { get; set; } = new List();
}
[Table("TemplateImages")]
public class TemplateImage
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public int Image1 { get; set; }
public int Image2 { get; set; }
public int Image3 { get; set; }
public int Image4 { get; set; }
public int Image5 { get; set; }
[ForeignKey(typeof(Template))]
public int TemplateId { get; set; }
}
In my application every template has 5 images, but this might change in the future. What is the correct approach according to DB design, to have a TemplateImage object with 5 images or to have only one image for every TemplateImage object like this:[Table("Templates")]
public class Template
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public int Category { get; set; }
[OneToMany]
public List Images { get; set; } = new List();
}
[Table("TemplateImages")]
public class TemplateImage
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public int ImaExoskeletor wrote:
but it's not working and i'm guessing it is not working because you cannot query with parameter "Images", in the database is stored as a blob. So im guessing i have to convert List to something else that is more easily supported in sqlite.
Comparing blobs isn't very efficient; make a hash-value of the blob (like md5), store that with the blob, and compare it using that. A hash-value would be like a fingerprint of your blob; it will change as soon as the image changes, but will result in the same values for the same images.
Exoskeletor wrote:
In my application every template has 5 images, but this might change in the future. What is the correct approach according to DB design, to have a TemplateImage object with 5 images or to have only one image for every TemplateImage object like this:
I'd have a table for the images; one field for the image, another field that holds the id of the template. That way you can have any number of images linked to the template.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
Exoskeletor wrote:
but it's not working and i'm guessing it is not working because you cannot query with parameter "Images", in the database is stored as a blob. So im guessing i have to convert List to something else that is more easily supported in sqlite.
Comparing blobs isn't very efficient; make a hash-value of the blob (like md5), store that with the blob, and compare it using that. A hash-value would be like a fingerprint of your blob; it will change as soon as the image changes, but will result in the same values for the same images.
Exoskeletor wrote:
In my application every template has 5 images, but this might change in the future. What is the correct approach according to DB design, to have a TemplateImage object with 5 images or to have only one image for every TemplateImage object like this:
I'd have a table for the images; one field for the image, another field that holds the id of the template. That way you can have any number of images linked to the template.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
Wow amazing response, thank you for your fast reply. Im thinking if it is overkill now to create a Table for the images rather than using the blob but i like the idea of table for the images because it feels right in terms of database design. However if i have a table for the images (so if i have a List) then i will not have a blob of a List to make a md5 hash of it right? In that case if i want to check if a template with the same images exist i would have to retrieve all the images and check them one by one. If there is a way to md5 the List it would be the best i think. I found also this c# - Create Hash Value on a List? - Stack Overflow[^] maybe i can use this to hash a List?
-
Wow amazing response, thank you for your fast reply. Im thinking if it is overkill now to create a Table for the images rather than using the blob but i like the idea of table for the images because it feels right in terms of database design. However if i have a table for the images (so if i have a List) then i will not have a blob of a List to make a md5 hash of it right? In that case if i want to check if a template with the same images exist i would have to retrieve all the images and check them one by one. If there is a way to md5 the List it would be the best i think. I found also this c# - Create Hash Value on a List? - Stack Overflow[^] maybe i can use this to hash a List?
Exoskeletor wrote:
i would have to retrieve all the images and check them one by one
you would retrieve all the hashes, not the images, compare the hashes, not the images. You would only get the images if you need to display them, everything else would use the hashes. You could also use the database to compare the hash using a where clause on your sql query.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
-
Exoskeletor wrote:
i would have to retrieve all the images and check them one by one
you would retrieve all the hashes, not the images, compare the hashes, not the images. You would only get the images if you need to display them, everything else would use the hashes. You could also use the database to compare the hash using a where clause on your sql query.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
Isn't it faster to hash the whole list instead of comparing 5 hashes with a set of other 5 hashes?
-
Isn't it faster to hash the whole list instead of comparing 5 hashes with a set of other 5 hashes?
Yes, but the business case would need to state that all 5 images must be identical.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
-
Yes, but the business case would need to state that all 5 images must be identical.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
You mean that it is better in terms of programming design to manually check each image and not the list of images? images are stored by their resource id which is unique, i could use this as a hash right?
-
Exoskeletor wrote:
i would have to retrieve all the images and check them one by one
you would retrieve all the hashes, not the images, compare the hashes, not the images. You would only get the images if you need to display them, everything else would use the hashes. You could also use the database to compare the hash using a where clause on your sql query.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
i would need 5 queries to check if the 5 images exist in the DB, this is not expensive?
-
i would need 5 queries to check if the 5 images exist in the DB, this is not expensive?
This decision cannot be driven by programming requirements but is a use/business case requirement. Do you need to know if all of the 5 images are identical between records or do you need to know if one of the 5 images is different. Does the sequence of images impact on the differences of the group.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
-
This decision cannot be driven by programming requirements but is a use/business case requirement. Do you need to know if all of the 5 images are identical between records or do you need to know if one of the 5 images is different. Does the sequence of images impact on the differences of the group.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
i prefer to know if all of them are identical, sequence is important when the app is running ,im getting them in the sequence i want with this code:
public static int GetSequenceHashCode(this IList sequence)
{
const int seed = 487;
const int modifier = 31;unchecked { return sequence.Aggregate(seed, (current, item) => (current \* modifier) + item.GetHashCode()); } }public static void AddTemplate(int category, List images)
{
var tmpl = new Template()
{
Category = category,
};
var img1 = new TemplateImage()
{
Category = category,
Image = images[0],
};
var img2 = new TemplateImage()
{
Category = category,
Image = images[1],
};
var img3 = new TemplateImage()
{
Category = category,
Image = images[2],
};
var img4 = new TemplateImage()
{
Category = category,
Image = images[3],
};
var img5 = new TemplateImage()
{
Category = category,
Image = images[4],
};
tmpl.TemplateImages = new List() { img1, img2, img3, img4, img5 };
tmpl.ImagesHash = tmpl.TemplateImages.GetSequenceHashCode();
var result = DatabaseHelper.db().Query -
i prefer to know if all of them are identical, sequence is important when the app is running ,im getting them in the sequence i want with this code:
public static int GetSequenceHashCode(this IList sequence)
{
const int seed = 487;
const int modifier = 31;unchecked { return sequence.Aggregate(seed, (current, item) => (current \* modifier) + item.GetHashCode()); } }public static void AddTemplate(int category, List images)
{
var tmpl = new Template()
{
Category = category,
};
var img1 = new TemplateImage()
{
Category = category,
Image = images[0],
};
var img2 = new TemplateImage()
{
Category = category,
Image = images[1],
};
var img3 = new TemplateImage()
{
Category = category,
Image = images[2],
};
var img4 = new TemplateImage()
{
Category = category,
Image = images[3],
};
var img5 = new TemplateImage()
{
Category = category,
Image = images[4],
};
tmpl.TemplateImages = new List() { img1, img2, img3, img4, img5 };
tmpl.ImagesHash = tmpl.TemplateImages.GetSequenceHashCode();
var result = DatabaseHelper.db().QueryExoskeletor wrote:
however the GetSequenceHashCode doesnt work as expected, it gives different results on every run, so i will have to check with another code.
Try the md5 hash; it will be consistent between runs.
Exoskeletor wrote:
But looks very bad to me, will this have impact on performance? is there any other way for the same result?
You could combine those five queries into one. Something like "SELECT Id FROM TemplateImages WHERE ImageHash IN (@value1, @value2, @value3, @value4, @value5". If the values already exist in the table, such a query would give you their Id's.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
Exoskeletor wrote:
however the GetSequenceHashCode doesnt work as expected, it gives different results on every run, so i will have to check with another code.
Try the md5 hash; it will be consistent between runs.
Exoskeletor wrote:
But looks very bad to me, will this have impact on performance? is there any other way for the same result?
You could combine those five queries into one. Something like "SELECT Id FROM TemplateImages WHERE ImageHash IN (@value1, @value2, @value3, @value4, @value5". If the values already exist in the table, such a query would give you their Id's.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
What should i md5?
-
What should i md5?
The blob (the image). There's more than one way to create a hash, and there are different algorithms that generate different hashes. MS recommends using SHA256, but the idea is the same. See MD5 Class (System.Security.Cryptography) | Microsoft Docs[^] Once you can do that, you can save the hash with the image. If you want to check whether the image is already in the database, you hash your image and query the database to see if it is already there.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
The blob (the image). There's more than one way to create a hash, and there are different algorithms that generate different hashes. MS recommends using SHA256, but the idea is the same. See MD5 Class (System.Security.Cryptography) | Microsoft Docs[^] Once you can do that, you can save the hash with the image. If you want to check whether the image is already in the database, you hash your image and query the database to see if it is already there.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
Im not using a blob any more, i follow your suggestion and now i use this:
\[Table("Templates")\] public class Template { \[PrimaryKey, AutoIncrement\] public int Id { get; set; } public int Category { get; set; } //\[TextBlob("imagesBlobbed")\] \[OneToMany\] public List TemplateImages { get; set; } public int ImagesHash { get; set; } //public string imagesBlobbed { get; set; } } \[Table("TemplateImages")\] public class TemplateImage { \[PrimaryKey, AutoIncrement\] public int Id { get; set; } public int Category { get; set; } public int Image { get; set; } \[ForeignKey(typeof(Template))\] public int TemplateId { get; set; } }So the question is, how i can md5 List? (or another consistent hash)
Quote:
You could combine those five queries into one. Something like "SELECT Id FROM TemplateImages WHERE ImageHash IN (@value1, @value2, @value3, @value4, @value5". If the values already exist in the table, such a query would give you their Id's.
why every image to have a hash? what hash will they have? i dont understand, this is how i can check if one image of the new template im trying to pass, exist on a template from the DB
var result = DatabaseHelper.db().Query("Select * from TemplateImages where Image=?", images[0]);
If i can find a single query that can do that for all the images, i think im good
-
Im not using a blob any more, i follow your suggestion and now i use this:
\[Table("Templates")\] public class Template { \[PrimaryKey, AutoIncrement\] public int Id { get; set; } public int Category { get; set; } //\[TextBlob("imagesBlobbed")\] \[OneToMany\] public List TemplateImages { get; set; } public int ImagesHash { get; set; } //public string imagesBlobbed { get; set; } } \[Table("TemplateImages")\] public class TemplateImage { \[PrimaryKey, AutoIncrement\] public int Id { get; set; } public int Category { get; set; } public int Image { get; set; } \[ForeignKey(typeof(Template))\] public int TemplateId { get; set; } }So the question is, how i can md5 List? (or another consistent hash)
Quote:
You could combine those five queries into one. Something like "SELECT Id FROM TemplateImages WHERE ImageHash IN (@value1, @value2, @value3, @value4, @value5". If the values already exist in the table, such a query would give you their Id's.
why every image to have a hash? what hash will they have? i dont understand, this is how i can check if one image of the new template im trying to pass, exist on a template from the DB
var result = DatabaseHelper.db().Query("Select * from TemplateImages where Image=?", images[0]);
If i can find a single query that can do that for all the images, i think im good
Exoskeletor wrote:
So the question is, how i can md5 List<templateimage>? (or another consistent hash)
Not the list, but the items in it; for each image, you want something that represents it so you can compare the item to other items. If you were to load each image and compare them pixel by pixel, the proces would be slow. Getting a fingerprint for each image, you could compare those - they're a lot shorter. If each image has a fingerprint stored, than you can check if the same image is there by checking if its fingerprint is there.
Exoskeletor wrote:
If i can find a single query that can do that for all the images, i think im good
I gave an example of that in the previous post. You'd need a query that checks for at least five values, no?
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
Exoskeletor wrote:
So the question is, how i can md5 List<templateimage>? (or another consistent hash)
Not the list, but the items in it; for each image, you want something that represents it so you can compare the item to other items. If you were to load each image and compare them pixel by pixel, the proces would be slow. Getting a fingerprint for each image, you could compare those - they're a lot shorter. If each image has a fingerprint stored, than you can check if the same image is there by checking if its fingerprint is there.
Exoskeletor wrote:
If i can find a single query that can do that for all the images, i think im good
I gave an example of that in the previous post. You'd need a query that checks for at least five values, no?
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
you gave an example with a imagehash. what is the imagehash in your example? how i can get a hash from the List or from the images?
-
Exoskeletor wrote:
i would have to retrieve all the images and check them one by one
you would retrieve all the hashes, not the images, compare the hashes, not the images. You would only get the images if you need to display them, everything else would use the hashes. You could also use the database to compare the hash using a where clause on your sql query.
Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP
i think i have an idea of how im going to do it, i can do something like this:
public static string GetMD5Hash(string text)
{
using ( var md5 = MD5.Create() )
{
byte[] computedHash = md5.ComputeHash( Encoding.UTF8.GetBytes(text) );
return new System.Runtime.Remoting.Metadata.W3cXsd2001.SoapHexBinary(computedHash).ToString();
}
}
var nums = new List {1, 2, 3};
var result = string.Join(", ", nums);
var hash = GetMD5Hash(result); -
you gave an example with a imagehash. what is the imagehash in your example? how i can get a hash from the List or from the images?
An example is Calculate MD5 Checksum for a File using C#[^]; that's assuming your image is a file - and it will return a string with the hashed value. Write a small project to hash a single image that is a file and that shows the result, then you can work from there. The resulting string will change whenever the content of the image changes. In your project, you'd have to compute the hash when you store the image in the database. Get to know the concept first, before trying to add it to an existing project :thumbsup:
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
An example is Calculate MD5 Checksum for a File using C#[^]; that's assuming your image is a file - and it will return a string with the hashed value. Write a small project to hash a single image that is a file and that shows the result, then you can work from there. The resulting string will change whenever the content of the image changes. In your project, you'd have to compute the hash when you store the image in the database. Get to know the concept first, before trying to add it to an existing project :thumbsup:
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
my images are xml files that exist within android app, with simple string code inside, but how i can use that to my case? if i hash every image then i will have 5 hashes, how i can store 5 hashes and check them when im inserting a new template with one query?
-
my images are xml files that exist within android app, with simple string code inside, but how i can use that to my case? if i hash every image then i will have 5 hashes, how i can store 5 hashes and check them when im inserting a new template with one query?
I think you are right, i have to md5 the file, and not the resource id which is generated on code compilation, thanks for that. i don't know how i can hash all of them and store them together as a single hash but for now only i will check only one image if already exist
-
my images are xml files that exist within android app, with simple string code inside, but how i can use that to my case? if i hash every image then i will have 5 hashes, how i can store 5 hashes and check them when im inserting a new template with one query?
Exoskeletor wrote:
how i can store 5 hashes
The same way you store the images. Add a string-column next to the blob itself. Calculate the hash from the blob, and write them at the same time.
Exoskeletor wrote:
and check them when im inserting a new template with one query?
You don't check during the insert, but prior. You weren't trying to compare images during the insert before; same applies here. You can check if any of the five hashes exists in the database with a single query though; but one problem at a time - it would work with five queries too, and you can rewrite to use a single query later on.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^] "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.