Posted on by gifguide2code

JSON: How to Read Your Google Location Data

Header

A few days ago, I was actively avoiding my life plans (and scrolling through WordPress) when I stumbled upon a GIF from Vincze Istvan showing where Google had plotted his locations over the last few years.

And my brain exploded. I mean, I knew in the back of my mind that anytime I checked my email or looked for directions or asked Lyft for a ride, my information at that particular time and place was probably being stored somewhere. That storage place had always felt far too abstract to make me paranoid.

Until I saw it projected in a GIF in a supremely tangible way.

So I followed the instructions to get my own data. You start out in Google Maps and hit the Menu.

To The Menu

Select “Your Timeline” from your options.

The Timeline

And behold. This is everywhere Google thinks I’ve been over the last five years or so.

My Google Locations

At first, I was horrified. Partially because the map really demonstrates how my ADD translated into major life decisions—old sublets, temp jobs, road trips. They’re all over the place. I’ve been a hot mess. Then I paused.

Silly Google, I thought, I’ve never been to Minneapolis. But then I clicked the red dot for details. Minneapolis-Saint Paul International Airport, it said. You had a layover on December 4, 2015, it said.

It was correct.

So there was my giant revelation for the week. Google remembers more about where I’ve been than I do.

What Does This Have to Do With JSON?

After embracing the horror, I immediately decided I wanted to see the raw data. Like, chop it up, poke it a little, and try to figure out why Google thinks I’ve never been to El Paso. I was literally there just a few days ago. No red dot. At this point, I’m still not sure what that’s about.

Back to the point.

You can download your location history with a few clicks.

Download Data

Then click next and follow the prompts. You can also grab a KML from here, but given the number of points, I’m thinking it might crash GoogleEarth.

So I downloaded the .json file, opened it in Notepad, and thought to myself, hey genius, you don’t know how to read JSON.

Hold up. What is JSON?

I’ve been dancing around JSON since I started learning about scripts (mainly web scrapers) because it’s damn near everywhere on the internet. It stands for JavaScript Object Notation, which sounds gross and intimidating.

Don’t panic.

It’s just a way to store data. A JSON object can store strings, numbers, arrays, boolean values. It’s basically a digital closet organized into key/value pairs.

What’s a Key/Value Pair?

Think of the key as a descriptor of the data or value you want to store. If “Character” is the key, “Rainbow Randolph” could be the value. To make a JSON object, alls you got to do is pop the key/value pairs in curly brackets separated with commas.

{ "Character": "Rainbow Randolf", "Actor": "Robin Williams", "Theme Song": "Friends come in all sizes"}

Here’s where things get a little complicated. You can make an key with an array value to store a lot of associated data. Arrays are stored in regular brackets. Here’s an example of a movie key associated with an array of its characters.

{"Death to Smootchy": ["Rainbow Randolf", "Smoochy the Rhino", "Buggy Ding Dong"]}

And here’s where things can really start to screw with your head. You can nest JSON objects. That’s right. Key/value pairs within key/value pairs. So you can have a movie associated with an array of characters–and details about each of those characters in their own objects.

{ "Death to Smoochy": [{ "Character": "Rainbow Randolf", "Actor": "Robin Williams", "Theme Song": "Friends come in all sizes"}, { "Character": "Smoochy the Rhino", "Actor": "Edward Norton", "Theme Song": "Here comes the cookie"}]}

You just have to watch out for all the brackets. The more complex the get, the easier it is to misplace one. Google Location History is not on the happy end of the complexity spectrum.

JSON Format

What are the Google Location History Keys?

The keys are timestampMS, latitudeE7, longitudeE7, accuracy, velocity, (sometimes heading), altitude, verticalAccuracy, and activity.

TimestampMS is a string. Everything else other than Activity is a number. Activity is an array. The Activity array has four nested JSON objects, each with their own Activity key associated with yet another nested JSON object including Type and Confidence keys.

I’m assuming this is because Activity is determined by getting your position over four successive moments in time to figure out whatever the heck you were doing. My activities were categorized into “STILL”, “IN_ROAD_VEHICLE”, “IN_VEHICLE”, “EXITING_VEHICLE”, “vehicle_personal_confidence”, “WALKING”, “RUNNING”, “ON_FOOT”, “TILTING”, “IN_RAIL_VEHICLE”, “ON_BICYCLE”, and “UNKNOWN.”

I’m unclear on the difference between on foot and walking, also in vehicle versus vehicle personal confidence. The others I’m assuming are determined by how fast you’re moving. I genuinely don’t know what tilting is about. If you happen to find others, please post them in the comments because I’m curious.This maps to Las Vegas if you're wondering...

Notice that it starts off with { “locations”: [ meaning all your locations are within an array in that one value.

What’s with the Time and Coordinate Values?

You may have noticed timestampMs looks funky. It’s not a traditional date display with a day, month, and year.

There exists a thing called Unix time. It’s counting upwards from January 1, 1970. The numbers you see in timestampMS are the milliseconds that have passed since then. There’s a converter at www.epochconverter.com which also has a marvelous breakdown of how to get Unix time in different coding languages.

So for instance, my times in the nested JSON objects above are:

1518020678010 = February 7, 2018 at 4:24:38.010 PM GMT
1518020917050 = February 7, 2018 at 4:28:37.050 PM GMT
1518021168024 = February 7, 2018 at 4:32:48.024 PM GMT
1518021413969 = February 7, 2018 at 4:36:53.969 PM GMT

So basically one every four minutes for 12 minutes. The timestampMS in the JSON object holding the coordinates is less than a minute later.

1518021441000 = February 7, 2018 at 4:37:21 PM GMT

The latitude and longitude values might look familiar because it’s just standard decimal degrees without the actual decimal points.

Google Earth

That’s all I’ve got for the moment. I’ve got plans for follow-up on this one over the next few months.

Best of luck in your own Google Location History data dives.

UPDATE: Just posted a Python Regex tutorial on how to slice out timestamps, latitudes, and longitudes from this JSON data for a nice, clean CSV file. Visit What the Heck is Regex?

33 Replies to “JSON: How to Read Your Google Location Data”

  1. I’m curious if you could explain all the red dots? Recently, my husband downloaded my map history data using JSON, and according to all the dots, has concluded that I have cheated on him numerous times with numerous people. These dots on my map show homes/businesses that I’ve never been to before, and people I’ve never even heard of, let alone met. He has since kicked me and our son out of the house, and we have moved on, but it still bothers me that Google has placed me at several locations (sometimes for hours) that I have never been to. What exactly ate these red dots supposed to show?

    Liked by 2 people

    Reply

    1. My condolences on the marriage, Melissa. We’ve ALL been there. Rule #1 for your next spouse–hands off the JSON data. Your red dots are sacred. They must be guarded like gold codes.

      Better yet, marry a luddite. Herd cats. Move to the Kamchatka Peninsula. I doubt Google will find you there.

      To be honest, I’m still not sure what causes the red dots. They just appear. Like magic. If you ever get a decent handle on it, please shoot an explanation my way.

      Thanks for stopping by. I so rarely hear from people passing through.

      Liked by 1 person

      Reply

    2. Nobody knows what the red dots are? Ummm… ok. So those red dots are points where Google made an educated guess as to your location based on the available data. Reasons that the data might show locations that are nowhere near correct include:

      1. Stolen or lost phone (phone is with someone else or in a vehicle away from owner). This won’t explain events that happen over the course of months. A few days? Maybe, by then you’ve figured out that your phone is gone and bought a new phone.

      2. Towers with competing signal strength or some sort of conflict in their associated metrics causing the phone’s connection to hop back and forth between the optimal tower which may be blocked by landscape, and a far away tower which may have a line of sight to you at that time. When this happens it’s usually obvious because you’ll appear to be at one location, and then 10 seconds later you’ll be a mile away and then instantly back near the first location etc.. This makes really long straight lines which zigzag back and forth.

      3. Some virus or malicious application could have fabricated false gps data to be logged I guess, but I haven’t yet heard of such a thing. At least not one that substitutes fictional location data in place of the actual data. What’s more, it’s quite unlikely that the fictional location data would actually amount to a believable narrative of your actions; taking place in or near your home-town or places you have been and showing, for instance, that when you got off work on a particular day you stopped at a particular motel for two hours then drove to McDonald’s and then got home from work at 8:30pm…etc.

      But in all fairness, it’s important to point out that a little bit of information can be dangerous when it’s in the hands of someone who doesn’t have the full story or someone without the benefit of applicable training and experience.

      Liked by 1 person

      Reply

      1. Ed, darling–I love reading comments from people. Even the trolls. It shows someone, shockingly, in the vast regions of the interwebs has found my blog, which to be honest, I largely had to abandon ages ago because I started moving around too much.
        If it makes you feel any better, I’m pretty sure Melissa was a figment of my best friend’s imagination. She used to have fun by posting nonsensical comments on my blog, and though I deleted some, I left this one (as she never did confirm it was indeed her).
        Melissa–if you do happen to be real, let us know how you’re doing. More importantly, let us know how your red dots are doing. And if that whole cat-herding thing worked out.

        Like

        Reply

        1. Not sure if my reply made it through… I don’t see it there so… Reposting it:

          I found your blog quite helpful actually since it confirmed what I was reluctant to believe: That nobody has, as of yet, really developed a good app to address these situations where there’s a 250MB .json (which is a staggering, insanely enormous size when you look at what the .json format is usually used for) and it needs to be brought up in real time and easily navigated, broken up, exported, or played back.

          Again, there is Scarygami, I know. It is literally the only way to accomplish what I need and it works. But it’s command-line stuff which means you need to have all your dates and times predetermined. The only way I found to do that was to use “Theyhaveyour. Info mapped” (or whatever the URLese is) and write down a list of days I want to explore in detail and then to scarygami to make a few “single-day” jsons that won’t cause Google Earth to have a stroke.

          I think I just coined a term.

          URLese: (you-are-ell-eeze): The transformation of a name or sentence that results from it’s conversion to a URL. example: a business such as “Mary’s Doylies” might be marysdoylies.com in URLese.

          Thank you and good night

          Like

          Reply

        2. Got any ideas on how to convert this longitude from Gmaps timeline raw data:
          latitudeE7: 340001310 longitudeE7: 3451784343 (JSON file value)
          Should be:
          latitude: 34.00013 longitude: -84.3183

          Trolling for an answer. Any guidance is appreciated.
          Regards,
          BZRKS

          Like

          Reply

    3. Each timestamp has an “accuracy” value determined by the number of GPS satellites that were connected at that very moment, sometimes the accuracy can be off quite a bit. Location History Visualizer Pro displays larger red dots to indicate you could have been anywhere inside a larger red circle (sometimes over a mile in diameter). The problem with Google Maps is it tries to “guess” where you were based on earlier and later data, places the same size dot where it guesses you were, and gives no indication that the location has been calculated and is speculative. Google maps shows I was inside the airport in a restricted area; Location History Visualizer Pro shows a dot about 6 blocks in diameter that includes the gas station across the street from the airport that I was actually getting gas at.

      Like

      Reply

    1. Wow Melissa, that is absolutely brutal. On the other hand, it’s best to avoid people who are that controlling and assumptive by nature. I am curious about your mystery locations in your Google location history though, and where this data came from.

      I wonder if perhaps you live in a very rural area and/or shut of your GPS sometimes, in which case Google or your device will sometimes estimate your location based on various information like nearby cell towers.

      Or, if you’ve signed-in to another device with your Google Account at some point (maybe someone else’s device, maybe some app or website that the person rarely uses, so they don’t realize you’re signed in), in which case maybe *that* device’s location is being combined with your *actual* location data.

      Like

      Reply

    1. To be honest… I’m not sure, but I think so. I’ve tried turning it off on my phone, and the dots seemed to stop. But I use Google maps for directions, so immediately after turning my location on for that, more dots appear.

      Like

      Reply

  6. Thanks for the article. There isn’t much on the internet about this data and how to parse through it. For me I have found up to 6 different objects inside the “activity” array.

    I’m very curious as to why only sometimes there is activity recorded and why other times there is only timestampMs, latitudeE7, longitudeE7, and accuracy. Other times there is velocity, altitude, verticalAccuracy, and heading.

    If anyone has any explanations, let me know! I’m actually working on a couple visualizations based on this Timeline data for a course for graduate school.

    Like

    Reply

  8. Great blog.

    I’m about to start an analytics project for a service industry client, and one thing i’m thinking about is using Google location data (for delivery drivers) to develop some fleet metrics, this blog has been very useful toward that end.

    This morning I threw some of my own location history data into Tableau, to start to get an understanding of the dataset. Can’t seem to post screenshots here unfortunately, but i can see that this data i going to be very useful.

    — Glenn

    Liked by 1 person

    Reply

    2. Scarygami also works. At least I was able to use it to extract date ranges from a 250MB+ .json from takeout and output the extracted data as .kml files. If you convert an entire location history .json then get ready to wait a while and remember that the output file will be twice as large as the .json.

      “They have your info” is an amazing site but, unfortunately it doesn’t seem to work well these days. I have tried multiple browsers on multiple computers and cannot get it to animate the activity. It used to
      do it fine though. So now I guess its .json (converted to)==> .kml, .csv, or .gpx and then imported into “My Maps” or “my places” or whatever it’s called – and even then I’m not sure that tracks can be animated unless you extract only a couple of days at a time.

      Like

      Reply

      1. i have several takeouts and theyhaveyour.info works well with only one of them. it looks like google has added to the info included in location json files or they changed something. only my oldest takeout location works well on theyhaveyour.info all the rest either dont load the info and none of the others give me the animated day visualizer. its so frustrating.

        Like

        Reply

  11. Here you can find the difference between ON_FOOT and RUNNING:

    int IN_VEHICLE The device is in a vehicle, such as a car.
    int ON_BICYCLE The device is on a bicycle.
    int ON_FOOT The device is on a user who is walking or running.
    int RUNNING The device is on a user who is running.
    int STILL The device is still (not moving).
    int TILTING The device angle relative to gravity changed significantly.
    int UNKNOWN Unable to detect the current activity.
    int WALKING The device is on a user who is walking.

    source: https://developers.google.com/android/reference/com/google/android/gms/location/DetectedActivity

    Like

    Reply

  13. What is “device tag”? It seems explanatory, your device’s tag number, but I can’t locate this number anywhere on my Samsung device. It’s not the serial or IMEI number. The device tag changed a few times during a 4 hour span so I’m wondering if another device was used. If so, how can I confirm this when I can’t find it? I got the json text-thing down, with the exception of this one value that I can’t find physically on the device(s) or on any online forums. It seems so simple but where the heck is it?!

    Like

    Reply

  14. I know the original question was a few years ago but I thought I would chime in. There is nothing more frustrating than using the coordinates google maps gives you in a data file, just to put them into Google Maps, and they do not work! Grrrrr! This is how you format GPS coordinates to get them to show the location. In my JSON file, it shows: “latitudeE7″: 461773421,”longitudeE7”: -1034064879. This is how they need to be formatted:

    46.1773421, -103.4064879

    It may seem self-explanatory but I shall explain. I obviously erased the “longitudeE7” and “latitudeE7” I also entered a period after the 2nd digit of the Latitude and entered a period after the 3rd digit of the Longitude. A few other key components are the comma after the Latitude, a single space separating the L and L, and the negative sign in front of the Longitude i.e. XX.XXXXXXX, -XXX.XXXXXX. Now, this isn’t always foolproof. You may have to shave a few digits off to get an actual location. It appears, at a minimum, you must have at least this many digits in the coordinates. 44. 2, -103.123 to gain a location. Hope this helps!

    Like

    Reply

  15. Question: will JSON files be more accurate than the timeline shows?

    I am having difficulty with matching my timeline with the JSON file I downloaded. Sometimes the timeline shows the day as: ” No visits for today” and I really didn’t leave but, other days say my address as 1 visit, and sometimes the timeline shows I left when I was at home ( but that day matches the previous day exactly). Also, my timeline will change what it says periodically and almost “update” so to speak.

    So, the JSON file of the timeline I downloaded, will it be telling me more accurately where I was?

    Thanks for helping 😁

    Like

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

%d bloggers like this: