QCon London - Elasticsearch and the Citizen Advice Bureau (CAB)
Elasticsearch powers the Citizen Advice Bureau (CAB) to monitor trends in society before they become issues.
The DataKind UK team is assisting the CAB to make sense of online usage of their services and in-person visits to their centres. They have more centres in the UK than Tesco has shops and data going back 10+ years of every person they assisted classified by problem type and geo data.
Datakind UK took 3 types of data: 1. All of their Google Analytics data on their advice guide website (a self-help version of going into one of their offices) 2. The records of all the physical office visits (~2M people and 6M issues per year), which are essentially just a date, an office ID, and the issue code the person was seen for. 3. ~50K/year detailed write ups of critical cases. These have 6 text fields and about 40 demographic fields.
They indexed all of these data sets in Elasticsearch and normalised across all their fields, so that they were searchable across any of the common fields (date, location, issue code) and allowed deep exploration of the data types individually. They then built a Kibana 4 dashboard on top of all of this to allow CAB staff do the data exploration themselves.
On average, about 5,700 people walk into their local Citizens Advice Bureaux every single day; one of over 3,000 locations in communities and high streets across England and Wales. Every day about 69,000 people look for help through Advice Guide, the online advice service, and over 4,300 more reach CAB on the phone.