Adding AI Insights to Data with Google Gemini

Yesterday, Elizabeth Siegle, a developer advocate for CLoudflare, showed off a really freaking cool demo making use of Cloudflare’s Workers AI support. Her demo made use of WNBA stats to create a beautiful dashboard that’s then enhanced with AI. You can find the demo here: https://wnba-analytics-ai-insights.streamlit.app/

I found this incredibly exciting. I last looked at Cloudflare’s AI stuff almost an entire year ago ("Using Cloudflare’s AI Workers to Add Translations to PDFs"), and I haven’t quite had a chance to try it again, mostly because I’ve been focused on Google Gemini for my Generative AI work.

From an API/usage perspective, Cloudflare’s Workers are easy as heck (although I recently had an issue with them that turned out to be a very unique edge case), and you can see this in her code behind the dashboard here: https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights/blob/main/app.py. Scroll down to the generate_insights Python method and you’ll see it’s a simple POST with a prompt to get the results.

As I said, this was exciting as heck to me. Last week, my Code Break episode was focused on charting in JavaScript. In that session, I made use of Chart.js to create charts for a simple set of sales data. This sales data was a hard coded set of totals for four products over 12 months. You can see an example chart I built here: https://cfjedimaster.github.io/codebr/charts1/chartjs2.html. In case you don’t want to click, here’s the chart:

Inspired by Elizabeth’s example, I wanted to take this chart, and see if Google could get insights from it. Here’s what I built.

Version One

For the first version, I started off with an HTML page making use of the same chart as shown above, but with an added empty div to share insights:

<h1>Sales Data</h1> <div id="chartWrapper"><canvas id="myChart"></canvas></div><div id="result"></div>

On the client-side, the JavaScript is mostly related to Chart.js, but at the end, I’ve added a call to my server-side code to get insights and render it to the page:

document.addEventListener('DOMContentLoaded', init, false);let sales;let $ result;async function init() {	$ result = document.querySelector('#result');	let req = await fetch('./data.json');	salesData = await req.json();	let chartData = [];	let productNames = ['Apples', 'Bananas', 'Cherries', 'Donuts'];	for(let p of productNames) {		let data = {			label:p, 			data: salesData.sales.map(d => {				for(let product of d.items) {					if(product.name === p) return product.total;				}			})		}		chartData.push(data);	}	chartLabels = salesData.sales.map(d => {		return d.date;	});	const ctx = document.getElementById('myChart');	new Chart(ctx, {		type: 'bar',		data: {			labels: chartLabels,			datasets:chartData,		},		options: {		scales: {			y: {			beginAtZero: true			}		}		}	});	$ result.innerHTML = '<p><i>Getting AI insights into this data...</i></p>';	let insightsReq = await fetch('/insights', {		method:'POST', 		body:JSON.stringify(salesData)	});	let insights = await insightsReq.json();	console.log(insights);	$ result.innerHTML = marked.parse(insights.text);}

So far, nothing special. Do note that I’m passing the same sales data I used in the chart to my server. This is a sample of that data, just the first three months:

{	"sales": [		{ 			"date":"1/2024", 			"items": [				{ "name": "Apples", "total": 541 },				{ "name": "Bananas", "total": 218 },				{ "name": "Cherries", "total": 490 },				{ "name": "Donuts", "total": 451 }			]		},		{ 			"date":"2/2024", 			"items": [				{ "name": "Apples", "total": 558 },				{ "name": "Bananas", "total": 198 },				{ "name": "Cherries", "total": 452 },				{ "name": "Donuts", "total": 491 }			]		},		{ 			"date":"3/2024", 			"items": [				{ "name": "Apples", "total": 521 },				{ "name": "Bananas", "total": 312 },				{ "name": "Cherries", "total": 402 },				{ "name": "Donuts", "total": 645 }			]		}	]}

Alright, the fun part comes at the server. I’ll share a link to the complete source in a bit, but here’s the Gemini aspect:

const MODEL_NAME = "gemini-1.5-pro-latest";const API_KEY = process.env.GOOGLE_API_KEY;const si = `You provide insights on sales data. You should return 3 to 5 insights about the products sold and their trends over time.`;const genAI = new GoogleGenerativeAI(API_KEY);const model = genAI.getGenerativeModel({ model: MODEL_NAME,	systemInstruction: {		parts: [{ text:si }],		role:"model"	}  });async function callGemini(data) {	const generationConfig = {		temperature: 0.9,		topK: 1,		topP: 1,		maxOutputTokens: 2048,	};	const safetySettings = [		{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_NONE, },		{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_NONE, },		{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_NONE, },		{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_NONE, },	];		const parts = [    	{text:JSON.stringify(data)},  	];	const result = await model.generateContent({		contents: [{ role: "user", parts }],		generationConfig,		safetySettings	});	console.log(JSON.stringify(result,null,'t'));	try {		if(result.response.promptFeedback && result.response.promptFeedback.blockReason) {			return { error: `Blocked for $ {result.response.promptFeedback.blockReason}` };		}		const response = result.response.candidates[0].content.parts[0].text;		return { response };	} catch(e) {		// better handling		return {			error:e.message		}	}	}

In this case, my system instruction does all the heavy lifting. I’m literally just passing my JSON to Gemini as is… and it works really well. I honestly thought I’d have to rewrite it, perhaps in simple text with tabs and such to line it up, but it didn’t need any help at all.

Here’s the result:

Screenshot showing rendered chart and insights

I recognize that text may be a bit hard to read, so here’s a copy:

Here are some insights from your sales data:

  1. Donuts are on a constant rise: Donut sales show a clear upward trend throughout the year, ending the year with more than double the sales they had in January. This suggests a strong positive response to donuts, perhaps due to seasonal preference or successful marketing.
  2. Apples see growth, Bananas decline: Apple sales demonstrate overall growth across the year, peaking towards the latter months. Conversely, banana sales have steadily declined, particularly in the last quarter. Investigating external factors like pricing or availability could explain these opposing trends.
  3. Cherries fluctuate, but remain relatively stable: Cherry sales don’t show a strong upward or downward trend. They peak mid-year and experience some dips, but generally remain within a certain range, suggesting consistent demand.
  4. Potential Seasonality: There’s a noticeable dip in sales for almost all items in April, followed by an upswing in May. This could be due to seasonal factors influencing consumer behavior, or perhaps external events impacting that specific period.
  5. Consider Donut Promotions: Given the consistent success of Donuts, further promotions or exploring variations of Donuts could capitalize on their popularity and drive even greater sales.

Sweet! You can find the source here: https://github.com/cfjedimaster/ai-testingzone/tree/main/chartdemo

Version Two

So, I was just going to stop there, and then I recognized something. When I built the HTML for this demo, I had copied in a template that made use of Shoelace, my favorite UI library built with web components. I wasn’t actually using any of them in them in my code so the smart thing would have been to remove the dependencies. I didn’t. Instead, I looked at the Shoelace site to see if perhaps I could render the insights a bit nicer.

While looking, I came across their Carousel component, and I thought it would be nice to display the insights, one at a time, in larger text to be a bit more… bold? I’m not a designer, and I don’t play one on TV, but I figured it was worth a shot.

On the client side, I modified my code a tiny bit, making the assumption I would get an array back from the server:

let insights = await insightsReq.json();console.log(insights);let html = `<sl-carousel pagination navigation>`;insights.forEach(i => html += `<sl-carousel-item style="background: var(--sl-color-red-200);font-size: var(--sl-font-size-2x-large);padding:20px;">$ {i}</sl-carousel-item>`);html += '</sl-carousel>';$ result.innerHTML = html;

Converting my insights into an array was trivial – I simply made use of JSON schema in my call to Gemini:

const schema = {	"description": "A list of insights",	"type": "array",	"items": {		"type":"string"	}};

I still sent the same prompt, the only change was to my "config" object:

const generationConfig = {	temperature: 0.9,	topK: 1,	topP: 1,	maxOutputTokens: 2048,	responseMimeType: "application/json",	responseSchema:schema};

The results were pretty impressive I think:

New carousel display

I love how big and impactful the insight is. If this were a dashboard on display, you could add the autoplay feature to the carousel and have it change automatically.

I was happy with this, but the red color made me think a bit. I liked the red, but I literally just got it from copying and pasting Shoelace sample code. I wondered if I could do something different. Before I show you that, here’s the source for this version: https://github.com/cfjedimaster/ai-testingzone/tree/main/chartdemo2

Version Three

As I said, the red kinda bothered me, as even though it wasn’t a bright red, red usually implies a warning or negative item. What if I could get Gemini to quantify it’s insights into three sentiments, positive, negative, and neutral?

Turns out, this was incredibly simple – I just updated my JSON Schema and system instructions:

const schema = {	"description": "A list of insights categorized by positive, neutral, or negative",	"type": "array",	"items": {		"type":"object",		"properties": {			"insight": {				"type":"string",				"description":"The actual insight."			},			"sentiment":{				"type":"string",				"enum":["positive","neutral","negative"]			}		}	}};const si = `You provide insights on sales data. You should return 3 to 5 insights about the products sold and their trends over time. For each insight, classify the sentiment as either positive, neutral, or negative.`;

Nothing else changed on the server. On the front end, I slightly tweaked my code:

let insights = await insightsReq.json();console.log(insights);let sentimentColors = {	"positive":"green",	"neutral":"blue",	"negative":"red"};let html = `<sl-carousel pagination navigation>`;insights.forEach(i => {	let mood = sentimentColors[i.sentiment];	html += `<sl-carousel-item style="background: var(--sl-color-$ {mood}-200);font-size: var(--sl-font-size-2x-large);padding:20px;">$ {i.insight}</sl-carousel-item>`;});html += '</sl-carousel>';$ result.innerHTML = html;

Here’s an example of a positive report:

Positive donut sales

And here’s a negative:

Negative banana sales

Blue is used for neutral:

A neutral insight

There ya go. You can find the code for this here: https://github.com/cfjedimaster/ai-testingzone/tree/main/chartdemo3

I wish I could host these publicly, but I don’t want to incur charges for a simple demo. 🙂 As always, let me know what you think, and huge thanks again to Elizabeth Siegle for the inspiration!

Raymond Camden

Posted in: JavaScript

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.