Getting The Washington Post's daily news to mobile subscribers means staying on top of the latest dispatches from log analysis tools in real time.
When the Post's mobile app development team first deployed Sumo Logic's log analysis tool about nine months ago, Sumo's native data collectors couldn't keep up with the stream of data from up to seven Android and iOS apps.
"We would see sometimes up to a 40% to 60% drop rate on those log files, and then the system was always attempting to catch up, it was never current," said Hardip Singh, mobile development manager at The Washington Post.
One of the paper's engineers responsible for the Sumo integration worked with another engineer on the Sumo side to use Lambda to speed ingestion of logs into the Sumo Logic system.
"Since using Lambda, we have 100% success rate on getting those log files," Singh said.
Lambda proved more effective than Sumo Logic's native data collector because Lambda immediately recognizes small changes to data and forwards the change on to the Sumo Logic system, as opposed to data collectors which were much slower to ascertain what data had changed.
Hardip Singhmobile development manager, The Washington Post
"Before, it was more of a pull model where some process woke up, inspected the directory, took what's in the directory now and compared it to what was in the directory before," Singh said. Running through this diff process for tens or hundreds of thousands of log files was computationally expensive -- "with the volume of data that we're sending, and the desire to get a real-time picture, that pull model simply doesn't work," he said.
Log analysis tool with data ingestion at webscale speed
After AWS Lambda picks up files and sends them to Sumo, the log analysis tool offers reporting, dashboards, and alerting as well as the ability to run specific advanced queries as needed.
"We can look at issues on an individual device level" using anonymous ID numbers, Singh said.
The real-time log analysis combination of AWS Lambda and Sumo Logic helps quickly catch and troubleshoot performance issues, rather than having to wait two or three hours to identify whether there was an issue. Singh recalled a time when an image resizer wasn't applying proper compression to photos, resulting in slowed performance on the mobile app.
"We knew pretty much immediately, because we got these notifications that said, 'Hey, your average performance has exceeded what you set up as a threshold,'" he said.
The system can also troubleshoot glitches with subscriptions, so that a subscriber on the other side of the planet can have login issues resolved remotely by The Washington Post mobile app development team, which can see what's going on with the user's individual device in real time using Sumo Logic's log analysis.
APIs on the horizon
AWS Lambda and Sumo Logic work nicely as a team, but the team at The Washington Post continues to search for the most efficient means of monitoring apps. Certain functions, such as keeping track of application heartbeats, are more suited for direct integration into Sumo Logic through HTTP APIs offered by the log analysis tool.
"If [Sumo] came up with native app-side SDKs, that would be really helpful," Singh said. "My developers need to focus on news delivery ... I don't want my developers to be focused on log transfers."
Security monitoring in a hybrid cloud age
How microservices replace monolithic apps