Need a suggestion

simpledeveloper

Hi I have a table which has 100s of 1000s of records, in it there is a column Blob which is storing a huge json string. We got a new Tableau Reports developer, who is saying he is not able to use that Blob column to read some data from it, because he doesn't know how to handle Json in Tableau. So he wanted me to write a Service to convert that Json string into Excel document, means it would be huge Excel documents in 1000s of numbers, if not in 100s of 1000s, can somebody please suggest me, what solution can I provide him. I already wrote the Service to Convert but writing into 1000s of Excel files is awful, I am looking for any better approach which can be feasible for both of us. Thanks a lot in advance.

OriginalGriff

Point him at this: JSON File - Tableau[^]

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!

simpledeveloper

He is asking for converting that into XML I will just create an another table out of this Database and convert and post it over there, simple jSon would have been better but he is saying he doesn't know

OriginalGriff

Sounds like he needs to start learning how to do his job ... :sigh:

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!

Lost User

Store the data as a CSV text file; Excel opens it as a single sheet workbook.

It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it. ― Confucian Analects: Rules of Confucius about his food

Lost User

Here's a suggestion; tell him to learn. JSON isn't complex; it is textbased, human readable, with plenty of tools. My first was DBaseIV. Would you convert all to DBaseIV format if I told you JSON is too complicated?

Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Mycroft Holmes

simpledeveloper wrote:

We got a new Tableau Reports developer

Send him back, he does not know how to do the job you hired him for! Calling himself a developer is disingenuous.

Never underestimate the power of human stupidity - RAH I'm old. I know stuff - JSOP

simpledeveloper

No I didn't hire him, I neither have authority to hire him my friend

simpledeveloper

But yes I did send him the jSon now we are trying to write a generic SQL Query to read from jSon string instead of conversions, then he wanted me to help him in that I am trying using OpenjSon, do you know any idea how to read through jSon string which has objects embedded within other objects. Any idea you have my friend.

Richard Deeming

If it's Microsoft SQL Server, the documentation is pretty good: OPENJSON (Transact-SQL) - SQL Server | Microsoft Docs[^]

"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer

jschell

simpledeveloper wrote:

Hi I have a table which has 100s of 1000s of records

simpledeveloper wrote:

huge json string.

Excel limits. 32k maximum size for a cell 1,048,576 maximum rows 16,384 columns Excel specifications and limits - Excel[^] If you already have a 200,000 rows then a reasonable growth estimate would mean your solution would need to support 2,000,000 rows. And that is not going happen in excel. You didn't define 'huge' nor what is in that json has but see the cell size limit and column limit. Again if you are already pushing those then growth would suggest you will exceed it.

simpledeveloper wrote:

So he wanted me to write a Service to convert that Json string into Excel document,

Is this a one time job? On demand? Once a day? Once a minute? That will impact what is reasonable in terms of how long you your job can take processing this.

simpledeveloper wrote:

convert that Json string into Excel document

Is that the only problem that it is json? Basically the acronym 'ETL' exists for this very thing. So why not do the following 1. Create another database, do not try to do this in the existing one. 2. Create a job that incrementally processes the rows and flattens outs the data into the new database. 3. Then the other person uses that database, with the data nicely parsed out to do what they want. How to flatten it out? Well if the data is nice and stable then you could just create a table with the columns named appropriately. If however the json is includes lots of depth and is dynamic then you will need to create a metadata solution such as a table that stores a name, and type, and perhaps another table with values. The types allows you to handle arrays and other embedded types (circular). Keeping in mind that this problem is one you would have needed to deal with an the Excel idea any ways. This also allows the data to be kept up to date. Then it is up to them, not you, how often to actually pull the data.