Processing: How to Add CSV Data to Pretty Maps

The internet has a shit-ton of map data. I didn’t realize it until I started skimming through HTML on sites like Craigslist and Yelp. Yes, by the way, I am that weirdo who sits in coffee shops on Saturday mornings muttering at tags I don’t understand.

The point is, I want to be able to play with that data in Processing. The first step is just getting that information into the sketch and mapping it properly. That’s what I’m doing today… I acknowledge that I get fixated on projects with very few practical applications. It’s possible I need to get out more.

Suggested Materials:

Google Earth Pro:
Required for the background image and to double check coordinates. ArcGIS can be used too, but if I could afford a subscription, I wouldn’t be doing this in Processing.
Processing:
Required to actually pull all the information together and display it.
Airbnb Scraper:
Required for the .csv file of latitude and longitude. I’m working on scraping more data off other websites, but since I’ve already got the Airbnb script, I’m taking advantage.

Step 1: Import the Image

To get the background image, I took a screenshot in Google Earth, scaled it down to a reasonable size, and saved it as SFMap.png. Then in my Processing script, I hit Sketch, Add File, and picked out the image.The script to add it looks like this.

Step 2: Import the Data

The .csv file is easier to handle if it has named headers. My file has a  Lat, Long and Price column.

After the table is loaded, tablename.getRowCount() will return the number of rows. To retrieve the values, tablename.getFloat(Row#,”ColumnName”) is a safe way to return the latitude and longitude without rounding them into oblivion.

Step 3: Scale Down the Coordinates

Let’s talk about where I live for a moment. I’m in North America. Specific, right? Northern hemisphere. Western side of the world.That translates to a positive latitude and a negative longitude. For anyone else out there who was asleep during 6th grade geography, latitude corresponds to the y-axis. Longitude is the x-axis. Don’t get those confused. You know what happens to people who get those confused? They stay up until 3 in the morning screaming incoherently at Google Earth Pro because it keeps sending points to Antarctica.

We know that the image is 1103 pixels wide which represents approximately 0.238952636719006 degrees because that’s the difference between my western and eastern boundaries in Google Earth. To get the X value, we can get the difference between the western boundary (where x=0) and the longitude of the point. Scaling then just requires cross-multiplying and dividing. Keep in mind, the difference between a negative and a negative is negative (yes, to answer your question, I’ve been exceptionally dim this week and forgot to account for that the first several test runs). To get it to display on the Processing grid, you’ll have to turn the longitude array positive.

Almost the same process applies for the latitude except the image is 302 pixels tall and that represents 0.0516437133917975 degrees. Since Processing places y=0 at the top of the image, when we scale the length, we’ll have to get the difference between the northern boundary and the point’s latitude. Then cross multiply and divide.

Step 4: Create an Array

I know what you’re thinking. There’s no way in hell I’m going to sit here and run the math on every one of those points. This situation has LOOP written across it in giant neon bubble letters. So let’s pull each coordinate from the table, run the math, and shove the scaled values into a separate array.

Yeah. Ummm. I think there’s a more condensed way to do this, but I like the 3 step system. Declare it, create it, and then assign values.

Step 3 is where we throw in the loop to do the scaling before assigning the values.

Step 5: Draw Stuff

I went with a simple ellipse filled with white (255,255,255). You’ll notice below that my color parameters include a fourth value—that’s the alpha transparency. I like my dots with a little haze around them, so I added several ellipses with different transparencies. That’s completely customizable depending on how you want to color code.

The Code

PImage img;
Table table;
float[] lat;
float[] lon;

void setup() {
size(1103, 302);
img = loadImage("SFMap.png");
//Load points from the Airbnb csv
table = loadTable("Airbnb.csv", "header");
//Create arrays from longitude and latitude
lat = new float[table.getRowCount()];
lon = new float[table.getRowCount()];
//Start a for loop to scale down X and Y values into coordinate arrays
//This bit needs adjusting if you import a map at a different scale
for (int i = 0; i < table.getRowCount(); i++) {
lat[i]= ((1103*(-122.560608677635-(table.getFloat(i, "Lat"))))/0.238952636719006);
lon[i]= ((302*(37.912159028101655-(table.getFloat(i, "Long"))))/0.0516437133917975);
}
}

void draw(){
image(img, 0,0);
//Make everything borderless
noStroke();
//Run a loop the length of the lat array, starting at 0 and increasing by 1
for (int i = 0; i < lat.length; i++) {
//Make a 9 pixel circle with a 20/255 transparency value (really transparent)
fill(255,255,255,20);
ellipse(-lat[i], lon[i],9,9);
//Make a 6 pixel circle with a 80/255 transparency value (transparent-ish)
fill(255,255,255,80);
ellipse(-lat[i], lon[i],6,6);
//Make a 3 pixel circle with a 155/255 transparency value (kinda transparent)
fill(255,255,255,155);
ellipse(-lat[i], lon[i],3,3);
//Make a 1 pixel circle with a 255 transparency value (not transparent)
fill(255,255,255,255);
ellipse(-lat[i], lon[i],1,1);
}
}