Inform your personal fitness journey with Elasticsearch queries and Kibana

By

Alexis Roberson

illustration-elastic-search-gear-v2-1680x980.png

Backstory 

Three years after starting my fitness journey, I have the privilege of starting all over again thanks to my obsession with chocolate and pastries. Although these foods aren’t inherently bad, my goal is to make informed decisions based on data from my initial fitness journey. 

In this blog, we will use the Elasticsearch Python Client to send MyFitnessPal data to Elastic® in order to better understand the trends and behaviors surrounding our nutrition, with the end result being both a health dashboard in Kibana® as well as the ability to run Elasticsearch® queries against our data.

MyFitnessPal

MyFitnessPal is a calorie tracking tool used to log foods for each meal of the day. Although it’s really helpful in providing a detailed breakdown for each day, I wanted to take this a step further and receive all these same details based on a date range. I also wanted to search for specific foods from previous meal entries based on days, meal type, and macronutrient breakdown. These details can give a better picture of your overall nutrition goals and accomplishments over a given period of time.

What is Elasticsearch?

Elasticsearch is a data store that allows you to search and analyze your data. It has the ability to ingest and index both structured and unstructured data depending on your use case. And serves as the foundation for all other solutions within the Elastic Stack.

Creating the health dashboard

First, we’ll start by running a Python script that builds on the myfitnesspal-Python library.  We will use it to grab our data from MyFitnessPal and send it to Elasticsearch. 

Once your data is in Elasticsearch, you’ll be able to run queries and aggregations against it, as well as visualize it within Kibana. 

Here is how your Kibana dashboard should look at the end of this tutorial.

Prerequisites

Before we begin, you will need:

We will be using a library called Python-myfitnesspal to grab the data from our personal account. All the code from this tutorial is available here.

Data extraction to Elasticsearch


First, let’s install the Elasticsearch Python client

Python -m pip install elasticsearch

Next, we’ll need to install the myfitnesspal package.

pip install myfitnesspal

Since the myfitnesspal library uses cookies to login from the terminal, we will have to grant the terminal full disk access. Be sure to reverse this access once data has been extracted.

Ensure you grant access to a browser you’ve previously used to log in to MyFitnessPal.

Using this library, we will be able to access the food diary, exercises, measurements, and reports as shown here. In this post, we’ll be focused only on the diary entries. 

To establish a connection to our Elasticsearch cluster, we need to specify our Elastic Cloud ID and authentication credentials. I recommend creating a .ini file and accessing these credentials using a config parser.

from elasticsearch import Elasticsearch 
import configparser

config = configparser.ConfigParser() 
config.read('mfp_elastic.ini')

# Elasticsearch client instance 
es_client = Elasticsearch( 
    cloud_id=config['ELASTIC']['cloud_id'], 
    basic_auth=("elastic", config['ELASTIC']['password']) 
) 
es_client.info()

We’ll also need to import the myfitnesspal library to use the client to access meal entries.

import myfitnesspal

client = myfitnesspal.Client()

Now, you can pass in your date range using the get_date() function.

def main():
    start_date = date(2020, 10, 1) 
    end_date = date(2020, 10, 1)

Finally, we can send the data to an Elasticsearch index once we replace the index value with the name of the index that will hold all the meals for each day.

# send breakfast, lunch, dinner structs to ES 
def send_meals_to_elasticsearch(meals_diary_entry): 
    es_client.index( 
        index='{INDEX_NAME_FOR_EACH_MEAL}', 
        document=meals_diary_entry 
) 

# send each days overall calories and macros to ES 
def send_todays_total_to_elasticsearch(total_daily_entry): 
    es_client.index( 
        index='{INDEX_NAME_FOR_TODAYS_TOTAL_MACROS}', 
        document=total_daily_entry 
)

Now, simply run the code that will log in to MyFitnessPal and send the data to Elasticsearch.

Python myfitnesspal_to_elasticsearch.py

After running the Python code, we should have two new indices stored in Elasticsearch: 

  1. The alexis_myfitnesspal_diary_official_index index , containing the calories and macros for each meal
  2. The total daily calories and macros for each day, contained in the alexis_daily_myfitnesspal_total_index index

The output for the alexis_myfitnesspal_diary_official_index index should look something like this:

{"date": "2020-10-01", "meal_type": "breakfast", "total_meal_macros": {"calories": 607.0, "carbohydrates": 41.0, "fat": 38.0, "protein": 34.0, "sodium": 423.0, "sugar": 23.0}, "list_of_food_macros": [{"food_macros": {"calories": 120.0, "carbohydrates": 6.0, "fat": 11.0, "protein": 2.0, "sodium": 5.0, "sugar": 0.0}, "name": "Avocado"}, {"food_macros": {"calories": 210.0, "carbohydrates": 0.0, "fat": 14.0, "protein": 24.0, "sodium": 195.0, "sugar": 0.0}, "name": "Hickman's Farm Fresh Egg"}, {"food_macros": {"calories": 100.0, "carbohydrates": 1.0, "fat": 8.0, "protein": 6.0, "sodium": 180.0, "sugar": 0.0}, "name": "Kraft Shredded Cheese Mexican"}, {"food_macros": {"calories": 69.0, "carbohydrates": 17.0, "fat": 0.0, "protein": 1.0, "sodium": 2.0, "sugar": 15.0}, "name": "Medjoul dates Dates"}, {"food_macros": {"calories": 63.0, "carbohydrates": 16.0, "fat": 0.0, "protein": 1.0, "sodium": 1.0, "sugar": 8.0}, "name": "Banana Large 8\" 9\" Long"}, {"food_macros": {"calories": 45.0, "carbohydrates": 1.0, "fat": 5.0, "protein": 0.0, "sodium": 40.0, "sugar": 0.0}, "name": "Silk Coconut Milk Unsweetened Coconut Milk"}], "list_of_foods": ["Avocado", "Hickman's Farm Fresh Egg", "Kraft Shredded Cheese Mexican", "Medjoul dates Dates", "Banana Large 8\" 9\" Long", "Silk Coconut Milk Unsweetened Coconut Milk"]}

{"date": "2020-10-01", "meal_type": "lunch", "total_meal_macros": {"calories": 315.0, "carbohydrates": 31.0, "fat": 3.0, "protein": 42.0, "sodium": 628.0, "sugar": 4.0}, "list_of_food_macros": [{"food_macros": {"calories": 24.0, "carbohydrates": 6.0, "fat": 0.0, "protein": 0.0, "sodium": 0.0, "sugar": 4.0}, "name": "Homemade Baked Egg Plant"}, {"food_macros": {"calories": 140.0, "carbohydrates": 0.0, "fat": 1.0, "protein": 34.0, "sodium": 300.0, "sugar": 0.0}, "name": "Starkist Chunk Light Tuna Pouch"}, {"food_macros": {"calories": 151.0, "carbohydrates": 25.0, "fat": 2.0, "protein": 8.0, "sodium": 328.0, "sugar": 0.0}, "name": "Right rice Thai curry rice"}], "list_of_foods": ["Homemade Baked Egg Plant", "Starkist Chunk Light Tuna Pouch", "Right rice Thai curry rice"]}

{"date": "2020-10-01", "meal_type": "dinner", "total_meal_macros": {"calories": 345.0, "carbohydrates": 29.0, "fat": 3.0, "protein": 46.0, "sodium": 971.0, "sugar": 1.0}, "list_of_food_macros": [{"food_macros": {"calories": 15.0, "carbohydrates": 2.0, "fat": 0.0, "protein": 2.0, "sodium": 51.0, "sugar": 0.0}, "name": "Spinach"}, {"food_macros": {"calories": 180.0, "carbohydrates": 25.0, "fat": 2.0, "protein": 10.0, "sodium": 390.0, "sugar": 0.0}, "name": "Right Rice Cilantro Lime Net Carbs Vegetable Rice"}, {"food_macros": {"calories": 140.0, "carbohydrates": 0.0, "fat": 1.0, "protein": 34.0, "sodium": 300.0, "sugar": 0.0}, "name": "Starkist Chunk Light Tuna Pouch"}, {"food_macros": {"calories": 10.0, "carbohydrates": 2.0, "fat": 0.0, "protein": 0.0, "sodium": 230.0, "sugar": 1.0}, "name": "Cleveland Kraut Whiskey Dill Sauerkraut"}], "list_of_foods": ["Spinach", "Right Rice Cilantro Lime Net Carbs Vegetable Rice", "Starkist Chunk Light Tuna Pouch", "Cleveland Kraut Whiskey Dill Sauerkraut"]}

{"date": "2020-10-01", "meal_type": "snacks", "total_meal_macros": {"calories": 248.0, "carbohydrates": 31.0, "fat": 7.0, "protein": 21.0, "sodium": 263.0, "sugar": 4.0}, "list_of_food_macros": [{"food_macros": {"calories": 38.0, "carbohydrates": 8.0, "fat": 0.0, "protein": 1.0, "sodium": 13.0, "sugar": 3.0}, "name": "cascadian farms organic fire roasted sweet potatoes"}, {"food_macros": {"calories": 210.0, "carbohydrates": 23.0, "fat": 7.0, "protein": 20.0, "sodium": 250.0, "sugar": 1.0}, "name": "Oh Yeah One Protein Bar Dark Chocolate Sea Salt"}], "list_of_foods": ["cascadian farms organic fire roasted sweet potatoes", "Oh Yeah One Protein Bar Dark Chocolate Sea Salt"]}

And the output for the alexis_daily_myfitnesspal_total_index index containing the total macros and calories for the day should look something like this:

{'date': '2020-10-01', 'daily_macros': {'calories': 1515.0, 'carbohydrates': 132.0, 'fat': 51.0, 'protein': 143.0, 'sodium': 2285.0, 'sugar': 32.0}}

Next we need to validate that our data is present in our Elastic Cloud cluster. Once logged in, select the menu panel -> Dev Tools.

Let’s ensure we have the data for each meal by running the following search request.

GET alexis_myfitnesspal_diary_official_index/_search

Output: 

 "_source": {
          "date": "2023-7-26",
          "meal_type": "breakfast",
          "total_meal_macros": {
            "calories": 466,
            "carbohydrates": 64,
            "fat": 15,
            "protein": 27,
            "sodium": 273,
            "sugar": 33
          },
          "list_of_food_macros": [
            {
              "food_macros": {
                "calories": 16,
                "carbohydrates": 3,
                "fat": 0,
                "protein": 2,
                "sodium": 55,
                "sugar": 0
              },
              "name": "Spinach"
            },
            {
              "food_macros": {
                "calories": 30,
                "carbohydrates": 1,
                "fat": 3,
                "protein": 1,
                "sodium": 135,
                "sugar": 0
              },
              "name": "Silk Unsweetened Almond Milk"
            },
            {
              "food_macros": {
                "calories": 80,
                "carbohydrates": 2,
                "fat": 2,
                "protein": 15,
                "sodium": 80,
                "sugar": 1
              },
              "name": "Garden of Life Sport Organic Protein"
            },
            {
              "food_macros": {
                "calories": 121,
                "carbohydrates": 31,
                "fat": 0,
                "protein": 2,
                "sodium": 1,
                "sugar": 17
              },
              "name": "Banana Large Fresh Banana"
            },
            {
              "food_macros": {
                "calories": 150,
                "carbohydrates": 10,
                "fat": 10,
                "protein": 6,
                "sodium": 0,
                "sugar": 0
              },
              "name": "simply nature chia seeds chia seeds"
            },
            {
              "food_macros": {
                "calories": 69,
                "carbohydrates": 17,
                "fat": 0,
                "protein": 1,
                "sodium": 2,
                "sugar": 15
              },
              "name": "Medjoul dates Dates"
            }
          ],
          "list_of_foods": [
            "Spinach",
            "Silk Unsweetened Almond Milk",
            "Garden of Life Sport Organic Protein",
            "Banana Large Fresh Banana",
            "simply nature chia seeds chia seeds",
            "Medjoul dates Dates"
          ]
        }
      }
...

This index contains not only the total macros, that is the protein, carbs and fats, for each meal, but also the nutrient breakdown for each food item within each meal. Lastly, it provides a full list of the foods consumed in that meal.

Now, let’s check the index containing the total or overall daily calories and macros by running the following command.

GET alexis_daily_myfitnesspal_total_index/_search

Output:

{
  "hits": {
    "total": {
      "value": 213,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "9wBzQ4oBg_eLj9VFLbl4",
        "_score": 1,
        "_source": {
          "date": "2020-10-01",
          "daily_macros": {
            "calories": 1515,
            "carbohydrates": 132,
            "fat": 51,
            "protein": 143,
            "sodium": 2285,
            "sugar": 32
          }
        }
      },
   ]
}

As you can see in the above example, I consumed 1515 calories in total on Oct. 01, 2020. 

Although retrieving meal entries for a specific day is great, this can be done by simply clicking on a day in the MyFitnessPal app. A more useful way to explore our data is by searching for multiple days over a period of time to gain deeper insight into any possible trends.

Let’s say we wanted to view all meals for a full month. We can run a bool query with a given date range. In this case, I chose the month of December, but feel free to choose any month.As you can see in the above example, I consumed 1515 calories in total on Oct. 01, 2020. 

Although retrieving meal entries for a specific day is great, this can be done by simply clicking on a day in the MyFitnessPal app. A more useful way to explore our data is by searching for multiple days over a period of time to gain deeper insight into any possible trends.

Let’s say we wanted to view all meals for a full month. We can run a bool query with a given date range. In this case, I chose the month of December, but feel free to choose any month.

GET alexis_myfitnesspal_diary_official_index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
           "date" : {
             	"gte": "2020-12-01",
             	"lte": "2020-12-30"
           }
         }
        }
      ]
    }
  }
}

Output:

{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 114,
      "relation": "eq"
    },
    "max_score": 0,
    "hits": [
      {
        "_index": "alexis_myfitnesspal_diary_official_index",
        "_id": "hQB0Q4oBg_eLj9VFoLqY",
        "_score": 0,
        "_source": {
          "date": "2020-12-01",
          "meal_type": "breakfast",
          "total_meal_macros": {
            "calories": 369,
            "carbohydrates": 32,
            "fat": 18,
            "protein": 24,
            "sodium": 184,
            "sugar": 11
          },
          "list_of_food_macros": [
            {
              "food_macros": {
                "calories": 4,
                "carbohydrates": 1,
                "fat": 0,
                "protein": 0,
                "sodium": 25,
                "sugar": 0
              },
              "name": "Lily of the Desert Aloe Vera Juice"
            },
            {
              "food_macros": {
                "calories": 160,
                "carbohydrates": 9,
                "fat": 11,
                "protein": 6,
                "sodium": 0,
                "sugar": 0
              },
              "name": "365 Whole Foods Market Organic Black Chia Seeds"
            },
            {
              "food_macros": {
                "calories": 24,
                "carbohydrates": 4,
                "fat": 0,
                "protein": 2,
                "sodium": 78,
                "sugar": 1
              },
              "name": "Spinach Baby Spinach"
            },
            {
              "food_macros": {
                "calories": 40,
                "carbohydrates": 0,
                "fat": 5,
                "protein": 0,
                "sodium": 0,
                "sugar": 0
              },
              "name": "Carlson Wild Norwegian Cod Liver Oil"
            },
            {
              "food_macros": {
                "calories": 80,
                "carbohydrates": 2,
                "fat": 2,
                "protein": 15,
                "sodium": 80,
                "sugar": 1
              },
              "name": "Garden of Life Sport Organic Protein"
            },
            {
              "food_macros": {
                "calories": 61,
                "carbohydrates": 16,
                "fat": 0,
                "protein": 1,
                "sodium": 1,
                "sugar": 9
              },
              "name": "Banana Large Fresh Banana"
            }
          ],
          "list_of_foods": [
            "Lily of the Desert Aloe Vera Juice",
            "365 Whole Foods Market Organic Black Chia Seeds",
            "Spinach Baby Spinach",
            "Carlson Wild Norwegian Cod Liver Oil",
            "Garden of Life Sport Organic Protein",
            "Banana Large Fresh Banana"
          ]
        }
      },
...

As you can see, we get back all the meals for each day in the month of December starting with the 1st. 

Now let’s say we wanted to filter by meal type. We can search for all meal entries where the meal_type is set to snack.

GET alexis_myfitnesspal_diary_official_index/_search
{
  "query": {
    "match": {
      "meal_type": "snacks"
    }
  }
}

Output:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 212,
      "relation": "eq"
    },
    "max_score": 1.3442461,
    "hits": [
      {
        "_index": "alexis_myfitnesspal_diary_official_index",
        "_id": "-QBzQ4oBg_eLj9VFMLmf",
        "_score": 1.3442461,
        "_source": {
          "date": "2020-10-01",
          "meal_type": "snacks",
          "total_meal_macros": {
            "calories": 248,
            "carbohydrates": 31,
            "fat": 7,
            "protein": 21,
            "sodium": 263,
            "sugar": 4
          },
          "list_of_food_macros": [
            {
              "food_macros": {
                "calories": 38,
                "carbohydrates": 8,
                "fat": 0,
                "protein": 1,
                "sodium": 13,
                "sugar": 3
              },
              "name": "cascadian farms organic fire roasted sweet potatoes"
            },
            {
              "food_macros": {
                "calories": 210,
                "carbohydrates": 23,
                "fat": 7,
                "protein": 20,
                "sodium": 250,
                "sugar": 1
              },
              "name": "Oh Yeah One Protein Bar Dark Chocolate Sea Salt"
            }
          ],
          "list_of_foods": [
            "cascadian farms organic fire roasted sweet potatoes",
            "Oh Yeah One Protein Bar Dark Chocolate Sea Salt"
          ]
        }
      },
...

Not only can you search for specific meals, but you can also search for meals within a given meal_type in which a certain food was consumed. For instance, you could search for all the days you consumed black chia seeds for breakfast.

GET alexis_myfitnesspal_diary_official_index/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": { "list_of_foods": "Black Chia Seeds" }}
      ],
      "filter": [
          {"term" : {"meal_type": "breakfast"}}
      ]
    }
  }
}

Output:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 60,
      "relation": "eq"
    },
    "max_score": 7.762555,
    "hits": [
      {
        "_index": "alexis_myfitnesspal_diary_official_index",
        "_id": "PQBzQ4oBg_eLj9VFzbrS",
        "_score": 7.762555,
        "_source": {
          "date": "2020-10-28",
          "meal_type": "breakfast",
          "total_meal_macros": {
            "calories": 380,
            "carbohydrates": 11,
            "fat": 23,
            "protein": 33,
            "sodium": 220,
            "sugar": 1
          },
          "list_of_food_macros": [
            {
              "food_macros": {
                "calories": 160,
                "carbohydrates": 9,
                "fat": 11,
                "protein": 6,
                "sodium": 0,
                "sugar": 0
              },
              "name": "365 Whole Foods Market Organic Black Chia Seeds"
            },
            {
              "food_macros": {
                "calories": 80,
                "carbohydrates": 2,
                "fat": 2,
                "protein": 15,
                "sodium": 80,
                "sugar": 1
              },
              "name": "Garden of Life Sport Organic Protein"
            },
            {
              "food_macros": {
                "calories": 140,
                "carbohydrates": 0,
                "fat": 10,
                "protein": 12,
                "sodium": 140,
                "sugar": 0
              },
              "name": "Nice eggs Eggs"
            }
          ],
          "list_of_foods": [
            "365 Whole Foods Market Organic Black Chia Seeds",
            "Garden of Life Sport Organic Protein",
            "Nice eggs Eggs"
          ]
        }
      },

As you can see, there are 60 days in my index, where black chia seeds were consumed for breakfast. 

While we are searching on a list field, It’s important to note there is no dedicated array data type in Elasticsearch. Therefore, it is best to use the nested data types in cases where you want to be able to query objects in an array independent from each other.

Along with querying for specific foods consumed in a certain meal of the day, we can also view our total calories and macros for each day within a month by running the previous command against our alexis_daily_myfitnesspal_total_index index.

GET alexis_daily_myfitnesspal_total_index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
           "date" : {
             "gte": "2020-12-01",
             "lte": "2020-12-30"
           }
         }
        }
      ]
    }
  }
}

Output:

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 30,
      "relation": "eq"
    },
    "max_score": 0,
    "hits": [
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "kg50Q4oBSJjU8Is4oIRw",
        "_score": 0,
        "_source": {
          "date": "2020-12-01",
          "daily_macros": {
            "calories": 1632,
            "carbohydrates": 97,
            "fat": 69,
            "protein": 158,
            "sodium": 1075,
            "sugar": 33
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "hwB0Q4oBg_eLj9VFprrK",
        "_score": 0,
        "_source": {
          "date": "2020-12-02",
          "daily_macros": {
            "calories": 1622,
            "carbohydrates": 90,
            "fat": 73,
            "protein": 151,
            "sodium": 1263,
            "sugar": 29
          }
        }
      },

This is great preparation for the visuals we will create later on this tutorial using this data.

An important principle I’ve learned throughout my fitness journey is that I have to keep my calorie intake below my maintenance calories in order to continue to see progress.

So a helpful search in determining how long you’ve consistently accomplished this goal is by seeing the amount of days my calories fell below maintenance. In my case, my maintenance calories were around 1,800 and above. So any days where my calories were below this number would indicate that I was in a caloric deficit. 

Here, I’ll again use bool query to view all the days my calories fell within a calorie deficit.

GET alexis_daily_myfitnesspal_total_index/_search
{
  "query": { 
    "bool": { 
      "filter": [ 
        { "range": { "daily_macros.calories": { "gte": 1400 }}},
        { "range": { "daily_macros.calories": { "lte": 1800 }}}
      ]
    }
  }
}

Output:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 137,
      "relation": "eq"
    },
    "max_score": 0,
    "hits": [
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "9wBzQ4oBg_eLj9VFLbl4",
        "_score": 0,
        "_source": {
          "date": "2020-10-01",
          "daily_macros": {
            "calories": 1515,
            "carbohydrates": 132,
            "fat": 51,
            "protein": 143,
            "sodium": 2285,
            "sugar": 32
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "-gBzQ4oBg_eLj9VFNbnD",
        "_score": 0,
        "_source": {
          "date": "2020-10-02",
          "daily_macros": {
            "calories": 1509,
            "carbohydrates": 151,
            "fat": 52,
            "protein": 131,
            "sodium": 1710,
            "sugar": 28
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "_gBzQ4oBg_eLj9VFO7mn",
        "_score": 0,
        "_source": {
          "date": "2020-10-03",
          "daily_macros": {
            "calories": 1504,
            "carbohydrates": 135,
            "fat": 51,
            "protein": 132,
            "sodium": 1477,
            "sugar": 50
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "9A5zQ4oBSJjU8Is4QYOr",
        "_score": 0,
        "_source": {
          "date": "2020-10-04",
          "daily_macros": {
            "calories": 1518,
            "carbohydrates": 126,
            "fat": 65,
            "protein": 119,
            "sodium": 1481,
            "sugar": 31
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "AgBzQ4oBg_eLj9VFR7qI",
        "_score": 0,
        "_source": {
          "date": "2020-10-05",
          "daily_macros": {
            "calories": 1537,
            "carbohydrates": 152,
            "fat": 46,
            "protein": 126,
            "sodium": 2339,
            "sugar": 44
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "BQBzQ4oBg_eLj9VFTbrq",
        "_score": 0,
        "_source": {
          "date": "2020-10-06",
          "daily_macros": {
            "calories": 1497,
            "carbohydrates": 127,
            "fat": 56,
            "protein": 130,
            "sodium": 1614,
            "sugar": 30
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "-g5zQ4oBSJjU8Is4U4MH",
        "_score": 0,
        "_source": {
          "date": "2020-10-07",
          "daily_macros": {
            "calories": 1493,
            "carbohydrates": 144,
            "fat": 53,
            "protein": 120,
            "sodium": 2114,
            "sugar": 32
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "DgBzQ4oBg_eLj9VFWLr7",
        "_score": 0,
        "_source": {
          "date": "2020-10-08",
          "daily_macros": {
            "calories": 1508,
            "carbohydrates": 135,
            "fat": 54,
            "protein": 113,
            "sodium": 2121,
            "sugar": 33
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "EQBzQ4oBg_eLj9VFXrre",
        "_score": 0,
        "_source": {
          "date": "2020-10-09",
          "daily_macros": {
            "calories": 1502,
            "carbohydrates": 136,
            "fat": 53,
            "protein": 128,
            "sodium": 1253,
            "sugar": 42
          }
        }
      },
      {
        "_index": "alexis_daily_myfitnesspal_total_index",
        "_id": "_w5zQ4oBSJjU8Is4ZIMH",
        "_score": 0,
        "_source": {
          "date": "2020-10-10",
          "daily_macros": {
            "calories": 1518,
            "carbohydrates": 127,
            "fat": 60,
            "protein": 125,
            "sodium": 2598,
            "sugar": 39
          }
        }
      }
    ]
  }
}

Visualization with Kibana

Now that we understand our data a little better, we can create a health dashboard showing the trends in our data at a glance.

Under the menu tab, select Dashboard in the Analytics section. Then, click Create dashboard to the right.

It might also be helpful to add your Maintenance Calories to the dashboard to compare the rest of your data. Maintenance calories are the minimum amount of calories your body requires to maintain your current bodyweight. This is determined by metrics like your weight, height, and/or body fat percentage. If you aren’t sure about your specific maintenance calories, you can use this online calculator to calculate them.

After inputting your information, you should receive this calorie breakdown.

We can start with the maintenance calories for the Sedentary option. Let’s manually add this number to our dashboard and save the markdown control.

The last metric we will manually add is our Basal Metabolic Rate. This is the amount of calories we burn at rest. So if we did nothing for an entire day, this is the number of calories we are guaranteed to burn. We can grab this number from the previous online calculator we used located above the maintenance calories.

Now let’s create visualizations based on the data we stored in our indices. Be sure to choose the desired date range for your dashboard. In my case, I’ll be viewing the data from my initial fitness journey from October 1, 2020, through May 1, 2021.

Now let’s calculate the average amount of calories consumed over a given time period. As this is a simple statistic, it’s good to use a Metric Lens visualization. For this metric, we’ll be using the index containing our total calories per day.

Average Calories metric

Video thumbnail

As you can see, we used the alexis_daily_myfitnesspal_total_index index to calculate the average calories consumed over a given period of time. This can be helpful in determining whether the calories consumed on average were generally below or above your maintenance calories. 

Calories Over Time

Now let’s create a line graph that tracks my calories from October 2020 to May 2021.

For this graph, we’ll be using the index containing our total calories and macros per day.

Video thumbnail

This graph is helpful for identifying anomalies or spikes in my calories. Further context can be provided as far as whether these spikes correlated with specific holidays, which could attest for a huge surplus in calories. Perhaps, the spikes are a clerical error or the result of disproportionate percentages for important macronutrients such as protein.

In our next graph, we’ll add a line graph representing the trends for protein, carbs, and fats.

For this graph, we’ll be using the index containing our total calories and macros per day.

Macros Over Time

Video thumbnail

This graph can be helpful in comparison to the calories graph. You can use it to view the macro breakdown for days in which spikes in calories occured. 

Nutrition Table 

For the Nutrition Table, we’ll be using the index containing our total calories and macros per day.

Video thumbnail

Detecting the root cause of a spike in calories can be determined using the graphs, but in some cases having a tabular view of both calories and macronutrients allows for further investigation. 

For instance, the calorie line graph shows a spike on 2020-11-06. When I compare this spike with the macro table, I see a dramatic increase in my protein is the cause of the spike. After referencing this with my food diary, I was able to see that a protein source was accidentally inputted multiple times causing the spike. 

Total days in Caloric Deficit / Caloric Surplus

For the Caloric Deficit / Caloric Surplus graph, we’ll be using the index containing our total calories per day.

Video thumbnail

As you can see, we created a bar graph based on calorie ranges. Since I know both my Maintenance Calories as well as my Basal Metabolic Rate, I can determine the amount of days I was in a calorie deficit compared to the time I ate at maintenance. Lastly, I included the total days for my fitness journey by including all calories 1,400 and above. 

This can be helpful during plateaus, which are time periods where you experience little to no progress in fat loss or strength gain. These can sometimes occur after being in a caloric deficit too long. The above graph can be used to determine when it’s time to bring your calories back up to maintenance. 

Average Calories per Week

In this next graph, we’ll be visualizing the moving average for calories per week. For this graph, we’ll be using the index containing our total calories per day.

Video thumbnail

In this graph, we are able to view our average amount of calories per week. This again gives us a visual at a glance of where our calories generally fell over a period of time. As you can see, I spent almost five months eating below maintenance.

Top 10 Most Consumed Foods

Lastly, we’ll be creating a tag cloud to represent the top 10 foods most consumed over a given time period. For this visual, we’ll be using the index containing the individual meals.

Video thumbnail

Here, we have a tag cloud used to showcase my top foods consumed. This can be modified to include top 5, top 20, or whatever number you think is best. This can be helpful in understanding foods you should maybe decrease or possibly foods you should eat more of in your fitness journey.

Your completed dashboard should look something like this:

Limitations

Since this approach relies on a library independent of MyFitnessPal and uses cookies to log in, it is only a viable approach as long as the website allows access.

Future implementations 

Though the goal of this tutorial was to reflect meal entries for a single user, having data from multiple users will create data on a macro level, especially when considering geo locations, like the typical eating habits of people in a given region in comparison to others. Also, adding data from other apps that track exercise, sleep, or steps would add an additional layer in considering all contributing factors with regard to health.

Leveraging Elasticsearch to make better health decisions

Understanding the power of Elasticsearch starts with using it to understand real life data. Using Elasticsearch, we were not only able to search for specific foods and meals within a date range, but also visualize trends that can help inform future health and fitness decisions.

Ready to get started with tracking your own fitness journey with Elastic? Begin a free trial of Elastic Cloud.

Resources
The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Elastic Cloud 무료 체험판 등록하기

선택하신 클라우드 서비스 제공자에서 완전히 로드된 배포를 이용하세요. Elasticsearch 개발사로서 클라우드에 있는 사용자의 Elastic 클러스터에 기능과 지원을 제공해드립니다.

무료 체험판 사용 시작