Entries in Innovation (41)

Tuesday
Aug172010

Diagnosing Complex Applications - Answering the Tough Questions

DateTuesday, August 17, 2010 at 9:37AM

One of the constant items that I come across in my professional career is the one that typically starts with leadership within an organization about a very basic premise - what is going on with their business applications? This seems like a very straightforward question. However in truth it is not as easy as one would imagine.

Many companies have grown organically that while beneficial has some operational costs to consider. One of the most challenging happens to be managing complex systems. This includes appraisal and diagnosis especially triage in the case where applications critical to the business are having issues. 

Generally speaking almost all the organizations that I have had experience with typically have the same set of problems:

 

  • Missing or out-of-date metrics. One cannot measure anything if nothing has been defined. This is where most businesses fail. 
  • Threshold Goals. Once metrics have determined thresholds have to be defined. These are basic boundaries that determine 3 basic states: healthy, not-so-healthy, and in jeopardy. These can also be characterized as zones: green, yellow and red. These boundaries help establish what the business expects from their applications and operations.
  • Growth expectations. Businesses expect growth. However asking them to come up with an expectation to create a model is something most do not want to do. This is a tough balancing act typically around "planned" growth events. In truth if something does really well, all previous growth projections tend to be irrelevant since in essence the scale changes say from tens of thousands to millions. Regardless a growth model needs to be in place.
  • Holistic analysis. Only a very small handful of organizations really see this as a key practice for complex applications and systems. Most think of only a handful of elements not their entirety. It is absolutely essential to look at the complete spectrum of options and be able to analyze everything that can impact a business. This means hardware, software, network, web traffic, and user-based activity. All of it.

 

So why these basics? It actually comes back to my training at Toyota. In order to diagnose what is wrong, you need to know what is normal. So the basic process that I go through includes:

  • Get existing information. Whether it be from existing tools, logs, etc. It is important to get what is available.
  • Target data to answer key questions. These data points range in names from Key Performance, to Business Activity, to Business Metrics, etc. Yet their purpose is the same - identifying key elements that the business is looking for to answer their questions.
  • Identify what is missing. Invariably there are elements that are missing. These need to be identified and then tackled in order of precedence.

Following this basic formula holistic diagnosis and analytics can be automated and evolved over time.

So what sorts of scenarios does this cover? Some of the basic ones include:

  • Capacity. The company is going to have some major event and wants to know if they can handle it. This is not simply not just a question about any one part of a complex system rather the complete domain itself. Can it handle the extra users? Can it handle the traffic? Can it handle the business transactions? Can it handle the fallout? What is most likely to break? When? Where? How is that handled? All of these smaller questions are wrapped around the initial one. 
  • Triage and Diagnosis. Another very common issue is around problems that have impacted a business. Why is X problem happening? What are the symptoms? How are symptoms winnowed to potential causes? How are potential causes vetted to actual causes? What are expected impacts to potential solutions? How fast can potential solutions be turned around? How much can triage address vs. long term care? Being able to effectively manage all aspects of a problem enables the business to rapidly identify barriers to their growth and operations saving money, cutting costs and capitalizing on opportunities.
  • Business opportunities. With all the diagnostics in place, analysis quickly moves into business opportunity analysis. What are customers? What are the various business units doing? When are they doing? Why are they doing it? Is there something we are not doing? Is there something we can do better? Once a business has the ability to look at their system in a holistic manner all sorts of interesting patterns emerge that are of interest to any business leadership.

When presenting a business with this sort of proposal it is daunting and in many cases especially from the operations-side of the house considered redundant. However it is not to say that the analytics are designed to replace existing investments, rather it is a way to look at what exists and identify/plug gaps. 

For example many businesses have raw infrastructure data in the form of CPU, memory, disk, network activity, web traffic, etc. However this data is almost never compared to business application operations which track groups of activity in relation to one another. For example I have often ask an operations expert what is the link-traffic for a user who is inquiring about their account? They can give me all sorts of raw data but cannot put it together. If I ask the specific application expert they can tell me the path but not the application components. If I ask a developer, they can tell me the functionality and application components, but rarely can tell me the actual business case. When put in this light the problem is very clear: each domain is responsible for their individual area of responsibility. However in most cases there is no one to put them all together. 

Once put together a business can actually see for every business activity, it's impact to their technical infrastructure, personnel, and operations. They can also then put important business events such as quarterly close, specific product promotions, and anything else together and view a complete high-level view of what happens to their organization when that occurs: how busy is their application, how many users are actually assisting in the endeavors, are reports being executed before/during/after the event, how many business operations are being executed, what partners are being used the most, etc. 

Being able to answer tough questions means diagnosing and analyzing complex applications and systems in a different and more relevant perspective. It also means being open to the idea that while you may have existing tools and perspectives available, it does not mean you can answer tough questions.

AuthorAltan Khendup | CommentPost a Comment |
tagged , , , in , ,
Tuesday
Aug032010

Innovation Everywhere - Mobile, Gaming, Social! Oh My!

DateTuesday, August 3, 2010 at 11:34AM

Recently there has been a flurry of innovative advances on a variety of fronts that while not new in concepts, they have definitely started to make their make in the market.

Mobility is one of the most obvious forms of innovation that has fundamentally changed the game. Apple and Google are new yet they have had such a deep and meaningful influence that one can scarcely have discussions in the local conferences without coming across a conversation about these devices and advances in the mobile marketplace. Also it's not just the developers and handset manufacturers looking at this market with a new eye. Even the American carriers such as AT&T and Verizon are doing so with services such as smartphone payments and others. It is clear that businesses small and large are looking at this market much more differently than they did a mere 5 years ago.

Gaming is another major area of innovation. While traditional games such as Starcraft 2, World of Warcraft, Red Dead Redemption and others have legions of faithful followers, it is the changes as a whole in the gaming world that offers studies in innovation for other industries. Companies such as Valve's Steam gaming service is an excellent model demonstrating what people expect for paid software deliveries. Gaming platforms such as Microsoft Xbox whose ability to include services such as Twitter, Facebook and NetFlix demonstrates a greater diversity of offerings for existing platforms while at the same time fundamentally challenging long held views of what consumers expect platforms to be. Even games such as Zynga's Farmville who came onto the Facebook scene not too long ago has done well in demonstrating what casual gaming can do on a non-traditional platform.

Finally another area of innovation happens to be growing era of social networking. I simply call it social innovation. These are the various ways of discovering and engaging others along various dimensions of interest whether it be personal interests, professional interests, political interests, hobbies, news, or any combination. In this emerging area it is clear that consumers desire and expect to be able to consume what they want, when they want, and interact how they want to in a manner that they personally deem safe and secure. What one person thinks is perhaps too public may be considered highly restrictive to another. In this new digitally-based socially connected network, who you know is really a powerful factor. After all, it's not a site you are necessarily visiting, as opposed to people that individuals find interesting or value that drive the activity.

While there are certainly many other areas of innovation, all of them represent some universal truths about the marketplace: they are new ways of looking at old problems, the entrepreneurs in these areas are bold, brave and fast, and the consumers they are tapping into are not brand new generations but rather existing generations. These new innovations are really about how to reach people in ways that they understand, in meaningful and valuable ways, and simplifying significant degrees of complexity. 

It is certainly a wonderful and exciting time! 

AuthorAltan Khendup | CommentPost a Comment |
tagged , in ,
Sunday
Jul182010

Data As A Service - A Practical Viewpoint

DateSunday, July 18, 2010 at 9:14AM

In a recent discussion the topic of clouds arouse. Now many in the industry are still attempting to come to grips with the fundamentals of this concept. In this specific conversation we traversed down the thoughts of public vs. private, externally housed vs. internally housed, cloud-bursting, etc. Eventually we landed on the topic of services such as Infrastructure-as-a-Service, Software-as-a-Service, Platforms-as-a-Service and Data-as-a-Service.

We focused on Data-as-a-Service mostly because the professionals I were speaking to had issues around data and were curious as to how Data-as-a-Service or DaaS could help address many concerns traditional companies now face.

If one looks at the internal arrangement of any company that has been around for a bit, their data ecology is a mix of islands within the corporate environment: RDBMS like Oracle and SQL Server in a variety and number, file-based data such as Excel spreadsheets and the like, electronic communication such as email, etc. On these islands lay valuable pieces of data, keys, details about accounts, customers, strategic initiatives, etc. Historically speaking obtaining valuable information from these data islands spread throughout the ecology has been tremendously painful and labor-intensive requiring an organization to place significant investments into things such as data warehousing. While effective on a number of levels, in today's age of lightning-fast changes being able to get to valuable information that not only resides within but about the data ecology is absolutely essential to survival.

This is where the concepts of services particularly around DaaS are very powerful. From a high-level the DaaS allows an organization to not only have access to required information, but also places powerful discovery and self-evolving mechanisms that had not previously existed into the hands of the organization. There are a few key concepts that help make the DaaS work:

  • Storage: This can be reasonably small to very large. In truth DaaS need not truly focus on this as storage is usually addressed as Infrastructure-as-a-Service os IaaS. However practically speaking it is just a matter of understanding whether the goal of DaaS is to hook into an existing IaaS or act as a mesh over existing islands.
  • Meta-Data Dictionary: Everyone in IT and development knows what a data dictionary is. And equally many business people care less. However the idea here is to evolve from isolated or even an enterprise data dictionary into more of a Meta-Data Dictionary. The reason? Data dictionaries are what I view as instance-specific ways of making a definition, placing things into that definition, and interacting with that definition. Practically speaking it is how say one gets data within an Oracle database, or within an Excel spreadsheet, etc. When traversing the larger ecology they are tremendously ineffective, unable to handle rapidly changing contexts across extended domains. Meta-Data Dictionaries serve this purpose. They tap into the local dictionaries and extend them to include a much larger array of context across the organization to provide answers more rapidly while saving time and effort.
  • Meaning: The next major component of DaaS is meaning. It is something that is not as commonly spoken about. Meaning in the context of DaaS is the ability to consistently present not only information about what resides in the ecology but also about the ecology with regard to the domain of the organization. In a traditional relational database for example, issuing a query does not take into account the fundamental about meaning. The results of a single query does not mean anything by itself to the organization until it is placed within a broader context that not only spans the targeted island, but all the desired islands within the ecology. As the context grows so does the ever changing complexity of establishing meaning. Working in tandem with a Meta-Data Dictionary as opposed to individual data dictionaries, meaning can be quickly determined as a question is posed throughout the data ecology.
  • Discovery: This takes the paradigm of search and applies it throughout the DaaS ecology. Whenever events happen within the DaaS that affect the meaning as interpreted via the Meta-Data Dictionary, discovery adjusts to that by not only making older patterns available but newer ones as well. In this manner an organization is capable of discovering evolving patterns within their ecology as it relates to their business.
  • Living Data: Now this is old hat to many in the internet crowd but fairly new to organizations especially since it arises with the mention of DaaS. This concept means that all data elements as they change with respect to the data ecology are available immediately upon a user request. Practically speaking it appears to a consumer that data changes it's behavior as they interact with it. Examples include Twitter updates, Google Finance chart navigation or online banking activity from any bank. These instances not only process events, but as the events have impact on the ecology they are reflected back to the requester within seconds.
  • Services: A fundamental aspect of any "as-a-Service" model is the concept of services. The type, manner, number of, and management of services in a DaaS should not be underestimated. Services need to be simple, powerful and flexible to meet the needs of the organization. For a typical DaaS because of it's Meta-Dictionary it also has services related to Meaning and Discovery that provide far more value than traditional access services such as query.

I have had the opportunity and privilege to work on such a platform in my career. Having a reasonably strong background in databases I can say the transition was not an easy one. All the newer dimensions require significantly more consideration and realization than a simple database perspective. For example with a DaaS one can see the changing patterns of behavior and ask questions and gain insights into complex questions not really addressable before. In one of my previous experiences working at a large telecom the question arose about how long would it take for data elements of a particular marketing campaign to reach all the necessary parts of the organization. As with most typical organizations, the answer was not really precise since it was a culmination of asking each division and then aggregating the responses. In many cases the divisions were not 100% certain of the timing themselves. With my primitive platform in place, we were able to look up the information in a few minutes and provide a more comfortable, provable answer to the organization in a rapid manner. The cost savings along were well worth it; 2 mins of a single individuals time vs. 30mins for 1200 people of varying levels. Other questions such as how the ecology handles volumes, what volumes mean in relation to business operations, the amount and volume of meaning inconsistencies and what savings could be achieved are just some of the more typical operational questions. However with a DaaS in place, higher value insights can be gained such as missed opportunities for new products/services based on customer activity, competitive standing based on social responses and replies with regards to existing products/services, capacity planning for bursting or planned progessions, and many others.

It was at this point my colleagues were thinking it would take them years to build out a DaaS. I responded that a DaaS does take effort, but not necessarily time. It is an equal mix of the deep technology which would be a blend of building it and using vendor tools, and the expertise and knowledge of technical and busines staff. From a tools perspective, solutions such as those provided by vendors such as QuePlix for data virtualization and Kapow for integration to leverage existing domains can quickly get an organization with significant existing assets to DaaS basics very rapidly. The core characteristic is the commitment from the organization. Any undertaking such as DaaS is something fundamental to the culture not just a dalliance.

Then I pointed out the shifting landscape of competitive pressure due to the economic crisis. Those with stronger, valuable, flexible and more timely interactions with their data ecologies are the ones that typically engage their customers more meaningfully. Whereas those with less capabilities quickly find themselves losing opportunities to competitors. From a career standpoint many of the new technologies related to the DaaS such as cloud concepts, big data, distributed data, and the like are some of the most in-demand skills not just hands-on, but in management, deployment, architecture, etc. As more and more companies realize the value of DaaS along with other strategic approaches, they are moving to embrace them in order to stay competitive and survive.

AuthorAltan Khendup | CommentPost a Comment |
tagged , , , in ,
Sunday
Jul112010

Games Create Unlikely Innovation - Creativity is Where You Find It

DateSunday, July 11, 2010 at 9:50AM

Recently I had a wonderful conversation with a colleague about looking at opportunities to innovate from different perspectives. As with most discussions the high level part went very well. As we proceeded down to progressively lower layers of detail it became very clear that while high level agreement was easily obtained, the actual implementation details were not as easy. Very quickly at various points of the discussion statements appeared such as "Doing that [some degree of implementation] is not as easy you make it out to be." or "I am not sure how such things would be doable." or "The team would have a hard time actually doing something like that."

I halted the conversation and pointed out the trend I was seeing; most of the points were along why things could not be done rather than what could be done given what was available. I mentioned how creative and innovative people follow their passions and dreams, accomplishing what they can with what they have. I called this ability "rising to the highest levels within your own limits".

The most common and consistent examples of doing this are actually in video gamer communities. These are individuals who are doing what they love - playing video games. They are not getting paid to do it. In fact they are often times paying the gaming companies money for the privilege to play. They are under tremendous constraints: they do not have access to the source code, they do not have direct access to key designers, they do not have any direct say into the product other than community influence, and they are not always regarded as leaders as opposed to consumers.

However these games actively nurture creativity and innovation. The first and most common example is the ability to share information among members of the community at hand. Some of them come in the form of forums which most are familiar with. However some take on significantly more. One such endeavor would be for a World of Warcraft site called World of Logs. The site offers graphical analysis widgets of raw logs to the player community to create wonderful dashboards that display game specific critical information in dashboards that rival many professionally done sites for companies. The information is not only displayed in a fixed format, but along contextually relevant dimensions for the community such that the most important information can be quickly and easily found. The fact that developers passionate about a game have applied technical skills, development practices and social networking paradigms typically found in only some of the most innovative companies is truly remarkable. Also consider the fact that the site is heavily constrained: limited budget, limited time, limited people, limited resources, etc. as compared to their corporate counterparts. Their passion and high constraints results in a highly innovative and creative approach in a subject that interests them that rivals and arguably surpasses many corporate attempts at analytical delivery.

Another example happens to be knowledge sharing and collaboration. Again the context is a game called Dungeons and Dragons Online. Similar to World of Warcraft, DDO as it is commonly called, appeals to a different type of gamer. While smaller in size than World of Warcraft, the DDO community is no less passionate than other gaming communities and equally innovative in their solutions. One of the best examples of sharing can be seen in the site DDOWiki.com. The paradigm is the same as Wikipedia only applied to a game setting. Here the community discusses game updates, class strategies, crafting, questing, items, and many other topics. It is often referenced by the community as a stable source of information related to many subjects within the game. As it's popularity grows, community members continue to expand and grow the content of the site. What is very remarkable happens to the diversity in age of the community members that consume and contribute. While it is indeed true that many younger adults are present, so are in fact many older individuals ranging in ages from their 40s to their 60s. Clearly the ability for a wide range of not only ages but experiences to rally around a common source that they are passionate about and to contribute their bits of knowledge and share it with a larger population is very insightful and inspiring.

These two examples are by no means the only ones around. In fact there are many examples of such innovation across the web. They are not only limited to games but to many other subjects as well.

I ended our conversation by pointing out that these communities are not being paid a salary to do what they do. They are doing it because they love it. They work in very constrained conditions. However what they produce is of high quality, leverages important social networking dimensions of velocity and engagement that allows them to grow, and demonstrates practical application of what many companies would consider advanced product development concepts and practices.

Hence in an age when companies are looking for different and transformational solutions to their problems, to expand into markets, and reach out to new consumers one has to be careful about drawing lines and boundaries about what is possible. Look in unconventional places and what one will find are numerous examples of solutions whose reliance on "non-traditional" lines of thought allow them to address their problems in whole new ways. It is the ability to learn from these examples and incorporate them into one's viewpoint that can make the difference in thinking and acting differently than other professionals when confronted with challenging problems.

AuthorAltan Khendup | CommentPost a Comment |
tagged , , , in , ,
Monday
Jul052010

Being Innovative - It Takes Less Effort Than You Think

DateMonday, July 5, 2010 at 7:46AM

Over this Fourth of July weekend I was fortunate to spend at least one day with many colleagues and professionals whom I respect. After the discussions about family, finances and global topics ran dry we moved along to "shop talk" of sorts focusing on the various trends, market dynamics, technology and talent changes we have seen recently and what they meant to individuals.

One of the most common topics happened to be innovation. The general view was that innovation was seen as very complex and difficult concept to implement in any organization. Many of these individuals were seeking my advice on how to best innovate within their organizations or for themselves.

In my experience innovation means many things to people. A rather simple and easy way to be innovative in my experience is to see and act differently. This sounds easier than it really is for most professionals since most subscribe to a set of rules and conditions that have been honed through years of experience in the industry. For example, many professionals learn to look at options that they think can be delivered. This is pretty normal; look at your toolbox containing items such as budget, time, talent, material, etc. and then come up with a plan that is well known and has a high rate of "predictable success". 

"Predictable success" is a term I use as a measurement that most businesses use on tasks that they have a need to feel "good" about. It is something that you see project managers measure, directors use on pretty slidedecks, and investors look at with interest. This is not to say that this is incorrect or not needed, rather this measurement is not the kind used for innovation.

I want to point out that innovation is about failure. Not just once or twice but a lot of the time. This is one of the most singularly frightening aspects about innovation to an organization and individuals. Failure is not considered a good thing. It is considered a weakness. So rather than fail, most companies will put out solutions that are "good enough" based on what they think their customers want. This is where things get really fuzzy. Most companies are really bad at figuring out who their customer is, let alone what they want.

For example let's look at the iPhone. When it first came out many in the telecom and handset industry thought it was "just another smartphone". Sure it was pretty. However I had heard numerous professionals with decades of experience saying things like "I looked at it. Not impressed." or "Nothing new there. It will not do well." or "I do not think this thing from Apple is going to do well." Several years later the iPhone is a huge hit and considered highly innovative even in spite of the fact that many of it's features are not really that new.

Innovation is not about being 100% new, but rather doing something in a new way that delights people. That is why the iPhone does better than other smart phones. It is also the same reasoning that can be attributed to FaceBook or Google. I call this being "delightfully different". It also sounds like a characteristic but it is really an intrinsic measurement, one of the most visceral. If you do something that people really like it shows on satisfaction, engagement levels, commitments, sales, etc. If not, then these remain relatively flat.

For most professionals adhering to "predictable success" in delivering solutions means not being "delightfully different" and vice versa. Again this is mostly due to their years of indoctrination. However being innovative does not take a lot of effort. It does take an effort on the part of the professional to be open-minded. Closing off any avenue of thought means eliminating whole realms of possibilities. This is not what people have to foster. Rather they have to look at the challenge, then determine an ideal way to deal with it, then using their toolbox find a way to make their idea a reality.

A case in point happened on a recent project where we were assigned a proprietary toolset from a vendor. All the normal approaches were not delivering results and jeopardized the entire project. I and a few others looked at the problem differently. We totally ignored what the toolset was capable of doing. We instead focused on what we needed to get done. The basic problem was that what was in place was not capable of doing the required workload in the time we needed. The answer was simple from that view, get the stated units of work to complete in the timeframe requested. The challenge: the current vendor toolset did not do this. The solution was easy once framed in this light: extend the vendor toolset to do what we needed. The end result was a deliverable capable of not only meeting the initial expectations, but delighted the business, technology and operational folks since it met all of their requirements under huge growth expectations with minimal time and money. Where almost all professionals get stalled is not looking at the problem in the most basic of forms and being open-minded about how to solve it.

This is where I explain to professionals those "fancy" management concepts about innovation apply. Once the challenge has been identified the organization needs those "innovators" to come up with solutions. It may fail once, twice, a dozen times before the ideal solution is delivered but it will be delivered. 

Being innovative from a professional standpoint is as varied as their career objectives. I usually pose thought-provoking questions. For example: "Why use a document library? Why not a wiki?", or "Why use a file folder? Why not create discovery?", or "Why delete data, why not keep it all?" The answers are startling in terms of how engrained they are in tradition as opposed to possibility. 

So why be innovative at all? As I point out to so many of my colleagues it is about their career. Times have changed so much that what they consider normal is now obsolete. For example most system administrators within a legacy organization are used to dealing with maybe a dozen servers where their counterparts in more innovative companies are used to dealing with hundreds if not thousands at a time. The mindsets, the skillsets, the possibilities for businesses even in this narrow area is huge. Mobile development is totally different now than it used to be. Android and iPhone are now whole new career options as are the user design experience, application domains, and many other dimensions to mobile. It is so new, most being less than 4 years old, that one would think that being "new" would be a liability, but in fact the opposite is true. Those with more experience are finding it hard to adjust with their more innovative counterparts growing at the expense of their counterparts.

Professionals may not like the change in the market, however it is now an expectation from many employers. To compete now and in the future, professionals will now have to be innovative and the first step in that new world is with their own mindset.

 

 

AuthorAltan Khendup | CommentPost a Comment |
tagged , , in , ,
Page 1 2 3 4 5 ... 9 Next 5 Entries ยป