Well that was satisfying.
-
Quote:
In this case it doesn't matter, because the data formats are all known ahead of time. It's not generalized
That is what I thought too. I had rolled my own JSON parser because the data was in a set format. Did not think I needed a whole big JSON library. Everything was humming along nicely until I was about 110,000 reads in, then the website threw in a different format and it all went boom. The fix was easy, but what a pain. I was glad I was the only one using the code.
Within you lies the power for good - Use it!
JSON is a bit different than CSV though. My JSON is compliant.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
IP-API.com - Geolocation API[^] is a little web service to get a bunch of info about where you're at based on your IP (or an IP you give it) I use it for my little internet clocks to fetch the timezone since they don't have GPS. It can return values in CSV, JSON or other formats. Arduino already has
stream.readStringUntil(',')so parsing CSV seemed trivial. For quite awhile I was doing that. On a lark, I decided to swap it out to use JSON and my little JSON pull parser for embedded. The resulting code is actually a bit shorter than the CSV approach. And it doesn't take a lot of RAM because it's a pull parser (like XmlReader), and not a DOM I'm leaving it in JSON I love when stuff like that happens.Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Very neat. But let me hijack the discussion. :-) How accurate are those IP-to-geographical location mappers? I always laugh, when I'm playing an online game, and so-called "hackers" (who are typically too young to know anything about the original meaning of the word) "threaten" to reveal my physical address based on my IP. Apparently that makes me dumb for not using a VPN. I always congratulate them for finding out what city my ISP operates from. Which is nearly 500km away from me.
-
Very neat. But let me hijack the discussion. :-) How accurate are those IP-to-geographical location mappers? I always laugh, when I'm playing an online game, and so-called "hackers" (who are typically too young to know anything about the original meaning of the word) "threaten" to reveal my physical address based on my IP. Apparently that makes me dumb for not using a VPN. I always congratulate them for finding out what city my ISP operates from. Which is nearly 500km away from me.
It depends on your ISP. Obviously in your case it might be an issue, but it's usually accurate enough to get the time zone and regional weather and stuff.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
JSON is a bit different than CSV though. My JSON is compliant.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
I rolled my own JSON reader -- converting JSON to XML as it goes, because SQL Server has an XML datatype and functions. And every once in a while I would receive a corrupt JSON file (produced by a third-party product) and I would need to deal with it. In such cases I needed to report where the error occurred.
-
I rolled my own JSON reader -- converting JSON to XML as it goes, because SQL Server has an XML datatype and functions. And every once in a while I would receive a corrupt JSON file (produced by a third-party product) and I would need to deal with it. In such cases I needed to report where the error occurred.
I derived my parser from this, ultimately: JSON[^] Although I first converted a lot of it to regex for reasons.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
Quote:
In this case it doesn't matter, because the data formats are all known ahead of time. It's not generalized
That is what I thought too. I had rolled my own JSON parser because the data was in a set format. Did not think I needed a whole big JSON library. Everything was humming along nicely until I was about 110,000 reads in, then the website threw in a different format and it all went boom. The fix was easy, but what a pain. I was glad I was the only one using the code.
Within you lies the power for good - Use it!
I also have not used an off-the-shelf JSON reader. I convert the JSON to XML; I do not need "objects". If I had "objects" I'd just have to serialize them to XML anyway and that's a waste of resources. Additionally, I don't need to have the whole dataset live in memory at one time.
-
I derived my parser from this, ultimately: JSON[^] Although I first converted a lot of it to regex for reasons.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Yep, I have referred to that many times. All I need to do is load data from JSON files into database tables. My reader scans along, tokenizing the JSON, and checking whether or not it has located an item I want to send to the database.
-
Yep, I have referred to that many times. All I need to do is load data from JSON files into database tables. My reader scans along, tokenizing the JSON, and checking whether or not it has located an item I want to send to the database.
Yeah I do similar. I used my Visual FA project to render some state graphs, and then used those to hand roll a lexer/tokenizer in C++. I hand rolled it because even though I have a generator for C/C++ I do some extra things while lexing. Like when I'm scanning a number it parses the number into a double and a long long both at the same time. That way I get numeric values on those fields and I don't lose precision for very long ints. Since it streams it can process very long numbers, even though it will lose precision
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
IP-API.com - Geolocation API[^] is a little web service to get a bunch of info about where you're at based on your IP (or an IP you give it) I use it for my little internet clocks to fetch the timezone since they don't have GPS. It can return values in CSV, JSON or other formats. Arduino already has
stream.readStringUntil(',')so parsing CSV seemed trivial. For quite awhile I was doing that. On a lark, I decided to swap it out to use JSON and my little JSON pull parser for embedded. The resulting code is actually a bit shorter than the CSV approach. And it doesn't take a lot of RAM because it's a pull parser (like XmlReader), and not a DOM I'm leaving it in JSON I love when stuff like that happens.Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
LOL!!! A lot of websites think they can locate me that way - even Google thinks I'm in Los Angeles (well, Hawthorne, at least). Not even close... :-D
Will Rogers never met me.
-
Very neat. But let me hijack the discussion. :-) How accurate are those IP-to-geographical location mappers? I always laugh, when I'm playing an online game, and so-called "hackers" (who are typically too young to know anything about the original meaning of the word) "threaten" to reveal my physical address based on my IP. Apparently that makes me dumb for not using a VPN. I always congratulate them for finding out what city my ISP operates from. Which is nearly 500km away from me.
I know the most fun thing about it is how many map to some location in the middle of nowhere. The GPS coords for the geographical center of the US... Some farm in Kansas or something. And if it doesn't know? That's the default. Anonymous is headquartered out of an old farm house in Kansas. Heh.
-
Yeah I do similar. I used my Visual FA project to render some state graphs, and then used those to hand roll a lexer/tokenizer in C++. I hand rolled it because even though I have a generator for C/C++ I do some extra things while lexing. Like when I'm scanning a number it parses the number into a double and a long long both at the same time. That way I get numeric values on those fields and I don't lose precision for very long ints. Since it streams it can process very long numbers, even though it will lose precision
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
honey the codewitch wrote:
parses the number into a double and a long long
Because I'm loading to SQL Server, I let the server do that as appropriate. I just send XML and extract individual values via XPath and squirt them into columns.
-
honey the codewitch wrote:
parses the number into a double and a long long
Because I'm loading to SQL Server, I let the server do that as appropriate. I just send XML and extract individual values via XPath and squirt them into columns.
Mine is a generalized JSON parser. I use it for various things.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
