After joining AWS a few years ago, I found myself building and operating the largest distributed and scaled systems of my professional career. The following news sources that were previously interesting and useful became crucial for my day-to-day life.
InfoQ - Direct content primarily sourced from hand's on practitioners and reviewed by editors. Check out the Architecture & Design section.
DZone - Articles (70k+), Refcardz (200+), Guides (30+), and Zones (14 categories, nearly all relating to distributed systems at some level). While the Zones cut across many areas, I find Cloud, Big Data, and Database especially useful.
High Scalability - Weekly digest in the world of distributed/scalable systems plus some independent and paid content.
Thoughtworks Technology Radar (Platforms) - Thoughtworks has some great minds in the area of distributed software development, and their overall Technology Radar is updated semi-annually with their opinion of where certain areas in technology are moving. Bonus points - build a technology radar for your team!
All Things Distributed(AWS CTO Werner Vogels' Blog) - Recently a great deal of AWS region launches, but occasional and historical distributed/scalable content.
The Next Platform (partnered with UK-based The Register) - Enterprise vendor specific news source - "Offers in-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds."
These resources allow me to take others' learnings to enhance my mindset and think and act at 10x in terms of performance, scale, volume, variety, velocity, failures, etc. However, one doesn't need to be working at AWS to require this level of currency. Market expectations now demand that we build and operate (or at least think in the case of prototyping/early development) at this scale across every solution to deliver value to customers and stakeholders.
My offer to present this content to individuals and companies has been well-received and I had a chance to do that as well a few times and am continuing to share what I've learned originally as well as the ongoing discussions I've had in the community. Double cool! Note, this offer is still open!
Here's the latest version of the presentation from SlideShare:
I'm always looking for more feedback, so please feel free to reach out to me for engaging discussions in bridging the gaps that frequently exist between various stakeholders in a product development organization. It's my passion to help us all work #bettertogether!
Ten years ago, I made my first visit to Asia on a work trip to visit my new team in Hyderabad, India. It deserves a separate blog post as it helped me form many opinions on how to have distributed teams work through a bunch of experiences that didn't. Although I've had the chance to work with folks from this area of the world in the mean time, I haven't been back since.
I'll blend right in!
Before the novelty washes away and I become a "local" I wanted to put together a list of what really knocked my socks off on the changes I immediately noticed. I won't go into what hasn't changed (yet)--I'm still the only white guy that I've seen in a week, and the Muslim/Hindu mix (27% / 70%) still produces exotic awe in me to the point that my favorite time pass here has been staring slack jawed through the open air as we drive across town in an auto-rickshaw.
No Immigration Insanity
It's midnight sometime in June of 2005. I've been traveling for over thirty hours and am in Asia for the first time in my life. I voluntarily leave my business class cocoon of comfort and we are marched off the highly efficient and ordered German Lufthansa flight into an area that can only be described as a marketplace of free thought and expression when it comes to immigration bureaucracy.
Through bleary eyes and ears canals that are still full of whatever packs them at 30,000 feet, I witness a visual and aural cacophony of stamping, yelling, and officials (I hope those are officials) with guns forming and reforming lines in a room the temperature and size of a warehouse janitor's closet. Somehow, I made it to the front of the line and people kept cutting in front of me, with every mother's son and daughter yelling at me from behind in an exotic mixture of Hindi and English to "go, just go...ahhhh..don't let him do that..you stay there..stop! move back! move forward! acha acha ACHA! don't go. You have to push forward, don't let them get away with that!"
I finally took the plunge and wedged myself in the fray by the source of the loudest stamping soundand forgot my typical American politeness. It worked! I was at the desk. Mr. Stampy looked at me not once, but kept grunting something indecipherable...it was probably multiple somethings, but it sure sounded the same. I just kept saying "I'm sorry..pardon me? I don't understand" Every response was stamping, which I took as a good thing. When he stopped grunting, I got pushed forward by someone and no one was pointing a gun at me. I walked forward. I had arrived.
In 2015, it's now just like any other immigration desk, and honestly...it's disappointing.
Gone are the Undiplomatic* Ambassadors
We Americans (at least the U.S. variety) have cars hard-wired into our DNA to the point where my son popped out already negotiating what kind of lemon I'd be selling to him when he turns 16. Suffice it to say, I'm always on the lookout for every make and model of car and how they're different than back home.
Walking out of the airport in 2005, it seemed like I was falling back in time into what I envisioned was present-day
Cuba. Every single car not only looked the same, but appeared to have been built in the mid 1950s. The airport was lined with so many of these Ambassadors to and of India to the point that I thought there might be a presidential cavalcade (maybe, just maybe celebrating my arrival).
When I walked out of the new MUCH more modern airport in 2015, I didn't see a single one. They've been replaced with Tatas, Mahindras, Hondas, Suzukis, and many more. It makes it seem a lot less exotic here now.
*I've been referring to these cars as Diplomats since 2005 as well. Oops.
Traffic Lights That Work
I have been raving that NO ONE pays attention to the "stop lights" here in Hyderabad for the past ten years. When I describe the people going against the lights, and many times against traffic in the oncoming lanes, most Indians nod their heads in understanding. But now, my story must change as people really stop at the lights now! I'm guessing that this is due to the fact that most intersections have well-masked officers and some have cameras watching. I'm guessing that the latter don't really work, but the former does.
Perhaps, there's a catchy and highly effecitve advertising campaign like... "D.A.R.E. to stop on red!" or "Better to stop on red than dead."
Here's a recent drive proving we stopped!
Smart Phones Used Unsmartly
This phone went to India with me in 05!
It's not that mobile phones didn't exist when I was here last, heck I was rocking my Blackberry and
emailing and calendaring like a champ! I honestly don't remember locals having them although they probably used them as, you know, phones. I'm sure that this comes as no surprise to absolutely no one that EVERYONE has a smart phone here now. However the thing that really surprised me was HOW they were used.
Here are things that I have observed, but was too much slack in the jaw to take a picture:
Scooter drivers texting and talking while driving
All three riders including the driver of a motorcycle talking or texting (or buying delicious takeaway byrani online) while said 2-wheeler is in rapid transit
A driver and rider on a motor cycle passing the phone between each other, talking to someone while manuvering in highly congested (think 3 cars and 10 motorcycles wide) and weaving traffic
Of course, this perspective is somewhat tainted by living in an area where people put on such airs as banning use of phones while driving and making mandatory the use of child safety seats and adult safety seat belts. #NannyState
Call To Prayer
I'm certain that this one has been going on for a long time, and I just didn't notice it due to the fact that I was staying in the hermetically-sealed location where they hold up all the white people--western luxury hotels and hi-tech offices. As I'm staying with my wife's family this time, I can hear, smell, and see so many more rich and amazing things. One that I heard almost immediately after arriving to their flat my first day here was the Muslim call to prayer around 5:30 AM.
This city has a plethora of mosques and multiple calls to prayer can be heard five times a day. Being outside of the rhetoric going on back home, it's awesome to see Hindus, Muslims, Christians, and so many more, living their lives in an incredibly open, inclusive, and participatory democracy under the Indian constitution.
The goals of creating a roadmap start out well enough--let's give our customers and stakeholders a picture of the future and get them excited about what we're going to deliver in the future. However, the implementation generally fails to delight. They are developed in a silo, are considered by many to be a long-term contracted commitment, and are incomplete. They generally don't give visibility to the exciting and enabling technology and operations/devops changes that are necessary to achieve much of the innovation and major organizational goals. Beyond basic technical debt, these are large changes that must be made in concert between the business (product/customers) and the technology (development, devops, ops, security, etc.) teams and there is opportunity for awesome alignment!
At Socialware I was responsible for serving all of these groups and developed a passion for solving this problem as a whole. This presentation will cover what I learned and give participants real-world examples they can take away.
I just submitted the this to the Keep Austin Agile 2016 Conference. It's tough to get selected as this is a world-class event with a small acceptance rate. However, I'm really looking forward to sharing it if given the opportunity!
Here's the title:
Radical Roadmapping - Creating Synchronized Agile Product and Technology Roadmaps
Here's the abstract:
This presentation will discuss why a company would create and maintain three major artifacts (innovation roadmap, infrastructure/platform roadmap, and operations/DevOps roadmap) as well as the process to do so. Further it will cover how to synchronize them in order to move away from making OR decisions to making AND decisions that will please all stakeholders. It will also discuss key cultural changes that must be present in order to achieve maximum benefit from this approach and challenges experienced along the way to making this a reality at Socialware, a SaaS product company. Finally, this will include real world examples of the evolution of these roadmaps over 18 months that participants can take away and use as guidelines for doing so.
This concept is RADICAL as it is innovative in both its novel approach and ability to drive enormously positive organizational agility.
Of course, in Matt's usual energetic* style, there will be tangents, humorous self-deprecating references of learning (aka failure), and time for participants to describe how this would "never work" in their organization coupled with Matt's re-framing to help them understand how it just might.
*Best feedback comment ever received in his Keep Austin Agile 2015 presentation on Continuous Capacity Planning: "Man, this guy has been drinking way too much coffee for a 4:00 PM presentation!"
Next Steps
Now to create the content based on the real world examples and learning with the team at Socialware--I'll continue meeting with folks in the community, sharing what I have so far and refining it as I go. If you're interested in this concept, please let me know--I'd love to share my ideas!
I'm really proud to have helped push this through to the final stages and receive this critical patent that captures and protects our ability to serve our amazing customers in a way that no one else can.
"This patent really represents the core of our technology, business and purpose," said Matt Roberts, Socialware vice president of product development. "While the language of patents is complex, we make it our mission to assure that social media use and compliance are easy for every financial advisor and firm. The technology behind this patent makes that possible, which is why it's so valuable."
I was fortunate to be selected to present at this year's Keep Austin Agile on May 8, 2015.
Many product development teams struggle through “painful” planning meetings. No one seems to enjoy them but they are deemed necessary for various reasons that all start off with good intentions and critical needs. They typically do not harness the power of being responsive to change and following YAGNI (You Ain't Gonna Need It) and delaying decision until the last responsible moment concepts that are core to true agility (i.e. more waste than the Austin Community Landfill).
In this talk, Matt will provide an approach that he’s used for years to strike the delicate balance of building a product and development roadmap that can be used for long-term planning, short-term planning, and being extremely responsive to change. It’s also respectful of the Agile Manifesto Principles and Values. This approach should make planning an easier, if not fun, for everyone from Developers to Product Managers to Executives. It will include a simulation of sorts, that walks through a short- and long-term roadmap that responds to change as unexpected (pulled from REAL LIFE!) things happen.
Here's a direct link to the presentation. I'd certainly enjoy any feedback in the comments section below.
A "gift" from the team after my latest quality crusade
I had the opportunity to present this information at two Agile Austin QA SIG meetings in September and October 2014. It was a great opportunity to share some of what I've learned since joining Socialware. It's helping me to change the culture for the benefit of our customers and is a key part of "The Socialware Way". Before too long, I expect nearly everyone to say, "Quality is sexy!"
Background
While greenfield development efforts always have the promise of "doing it right" from the ground up, they can quickly devolve into the "legacy" systems that are essentially unsafe to modify as doing so will almost guarantee problems for users that the development team cannot anticipate. Further, many of today's applications exist in a complex and fragile ecosystem of APIs and other dependencies that are beyond the control of a team, division, or an entire organization. A culture of continuous learning is key in combating these challenges to create safe and valuable software for the customers and development teams that build and maintain them.
At Socialware, we have started the process of reviewing all production critical issues in an open and visible manner by using a Quality Circle approach as a team. We call this the Critical Defect Review Process. This is especially important due to the fact that our products exist on top of the constantly changing APIs of LinkedIn, Facebook, and Twitter--the world's largest social media companies. As our company has the largest Social Media deployments by Financial Services firms in the world, who have little, if any, tolerance for problems due to their scale and regulatory requirements, the focus on quality is paramount.
The Goal
The goal of the Critical Defect Review Process is to understand the root cause of why any high-priority defect exists in production and take specific remediation actions to ensure that this issue and others do not occur again. We also establish ownership of any actions going forward. Finally, we keep a recorded log of these issues, which will be reviewed periodically. Note, that the goal is NOT a "witch hunt" in terms of assigning blame. If we are not open to deep understanding, we will never be able to solve the problem. The ultimate goal of this process is to deprecate the process when we have zero critical defects in production, however this occurs.
Implementation
We continue to monitor this process and continuously inspect and adapt it, but currently, this is the basic framework of our Critical Defect Review Process.
Periodicty
This review occurs every three weeks (the same cadence as our sprints) with all resolved production defects marked as either a "Priority 1" or "Priority 2." The reason we wait until they are fixed is that the highest priority is to resolve any customer-critical issues, regardless of their genesis. Additionally, reflection generally helps towards the further understanding of the root cause of problems, and the eventual solution.
Participants
We have the following roles participate at these reviews:
Moderator/Scribe (generally a member of the Development Management team)
Product Technical Lead
Primary Engineer involved
Primary QA Engineer involved
QA Lead
Customer Support Lead
All team members are, of course, welcome to attend and listen
Format
The discussion captures the issue in question, any related issues, the primary engineer associated with the issue, the primary QA engineer associated with the issue, the root cause/synopsis, recommended actions moving forward (remediation), and agreed actions. All of this is captured in our wiki. Note, the agreed actions may be an acceptance of a certain risk within our process, although this is not the primary desired objective. This information will be reviewed at a regular basis by Product Development Management, Executive Leadership, and the team, at the very least during the sprint retrospective. Again, there is no blame assigned, which can only work within the larger context of "The Socialware Way" where our team members are respected, trusted, and treated as our most precious assets as opposed to "resources."
Early Results
While we have only been doing this for about two months, we've seen some fairly impressive results. Not only are the issues trending down, we are also able to respond more quickly and have a deeper understanding of why quality problems exist. Of course, we are implementing a number of other changes to our product development system that result in high-quality customer value as a natural outcome of our system rather than something that needs to be forced. So, the attribution of the increased quality is manifold. I will continue to monitor and share results.
Genesis
The genesis of this concept comes from the concept of a Quality Circle, which was introduced by Dr. Ishikawa. It was interpreted into "cross-functional teams" which was a key part of some Total Quality Management systems. Thoughts?
I would love to hear your thoughts on this process, please feel free to direct message, or comment below. Thank you for your time in reading!
