API code embedded directly in Data Cube using python Generator

#1

Hello,
[Google News API]
I’m trying to embedded Json API Code directly into Dundas, First i write some script to import data by using API Key and Url link to request and get data using Python data Generation. Successfully i got some data it showing in data preview as result, status and author.
below are the script i wrote and data preview what i got form that link.

image
image.png953×511 20.9 KB

Result was showing Source Code
image
image.png1445×247 7.1 KB

But, I need to Flatten that file to show like a columns and Rows(Table) format to use in dashboard. If any one have any idea how to do it. Just let me know your thoughts’ on it.

Thanks

#2

You need to get it into a pandas table. I’ve not done it with json myself, but there should be a json to pandas table method out there.

1 Like
#3

Hello Ramya,

Try modifying the last line of your code to something like this:

  • return json_normalize(data, record_path=[‘articles’]) *

This should flatten the articles column.
Let me know if this works for you.

#4

Thanks for reply!
I tried by modifying the last line but no use. It’s not accepting the record_path=[‘articles’] to flatten the column. And even data is not showing.

#5

I see, can you send a screenshot of the code after adding the record path portion.

#6

Sure,

image
image.png1526×559 109 KB

#7

image
image.png1694×493 107 KB

Data is not showing in data preview

#8

Thank you for reply,

ACtually i tried to convert that data into pandas table by using concept convert json to pandas DataFrame table method.
It’s showing some error please see that and if you know anything let me know.
This is the image i wrote the code in python analysis node-

image
image.png1298×502 17.6 KB

I got some error like this

image
image.png1239×619 106 KB

#9

Unless I’m mistaken, isn’t the problem here that data is undefined?

If you’re trying to access the input data I think you need to add the column as a placeholder. This process is outlined in the help article about the Python Analysis transform.

#10

Thanks for reply!

Actually i did same as in that article placeholder given for each column already. I just need to convert that data in data cubes or is there any possibility on dashboard also.

#11

Hello Ramya, apologies for the delay.
In this screenshot you sent the error message is obscuring the code. Can you double-check if there is an indent or space before your return statement? That could be a possible cause of the error.

#12

Thank you so much Lotachukwu. I got it and it changed into table. It’s all because of the spaces and place holder names. Now i got.

1 Like
#13

Great job Ramya. Glad it worked out.
I had a similar problem a while back (am relatively new to python just like you are). This was the referenced article that solution came from. Just in case you were curious and wanted to do some additional reading.
https://towardsdatascience.com/all-pandas-json-normalize-you-should-know-for-flattening-json-13eae1dfb7dd

2 Likes