The Technology Adoption Lifecycle

May 24, 2018

A while back I wrote about all the so-called Chasms. These days we begin our continuous innovations in the late mainstreet. Nobody crosses the Chasm.

I was working on watching a data from a pseudorandom generator for a normal distribution converge to a normal. That is supposed to happen by the time you have 36 data points. It didn’t happen. And, it didn’t happen by the time I plotted 50 data points. It didn’t help that I had to generate more data after the first 36 data points.

I made a mistake. Each call of the generator starts the process off with a new seed, aka a new distribution, so of course, it doesn’t converge. I’m not liking this dataset mindset of statistics. I’m not p hunting. I’m trying to validate a decision made in the Agile development process. I don’t have all day, but apparently, I have a week. Claims about fast discovery turn out to be bunk. A friend of mine suggested taking a Bayesian approach instead.

Through some, now forgotten thought process, I was plotting sigmas and z-scores, et all. That brought me back to some details of the technology adoption lifecycle (TALC). So I Googled it and found a whole lot of graphs of it that were just flat out wrong. No wonder everyone is confused about the Chasm. They are using one of the revised (wrongly drawn) figures. So I’ll show you some of the figures, point out the errors, and draw an older more correct view.

The misstatements seem to be sourced from Geoffrey Moore. When he moved into the late phases when the dot bust happened, he set about making the TALC relevant to the late phases and the biz orthodoxy. He has taken back most of the claims he made in his prior version of the TALC. It’s all disappointing.

One thing Moore said back in the beginning of his TALC, not Rodger’s version, was that it was was not a clock. I always thought he meant not an asynchronous clock, aka not like email. No, what he meant was we can choose to enter any phase we want. That leaves money on the table, but it accurately reflected what businesses do. This very characteristic means that businesses can completely skip the Chasm, the bowling alley, and his first tornado. Yes, some acquiring companies skip the second tornado or just suck at it so the acquisition fails. Mostly, acquisitions don’t even try to succeed. The VCs got their exit, that being the whole point of most VC investments these days.

Once you skip over the processes that are Moore’s contribution to technology adoption, people feel free to just fall back to Rodgers, a solely sociological collection of populations. Moore took Rodgers someplace else. Yes, Rodgers didn’t see the Chasm. But, Moore didn’t see Myerson’s Poisson games. The underlying model changed over time. I’ve modified the model myself. But, Moore’s processes didn’t move.

So let’s look at the mess.

01 TALC 2018

Figures from

  1. joshuafisher.com
  2. http://www.startify7.eu/wiki/technology-adoption-lifecycle
  3. software.homeaway.com
  4. wikimedia.org/wikipedia/commons/d/d3/Technology- Adoption-Lifecycle.png
  5. researchgate.net

I’m just citing the sources of the figures. They probably copied them from others that copied them. I’m not assigning blame. But, this very small sample demonstrates the sources of confusion about the Chasm.

Problems:

  • In figures 1, 2, 3, and 5, the first phase is called “Innovators.” Well, no. The inventors happened a long time before the technology adoption lifecycle began. The word innovators are indicative of management. In the earlier texts, this population was called technical enthusiasts. They are engineers, not business people. And, in the bowling alley and vertical sense, they were programmers known to the early adopter for the given vertical.
  • In figure 2, the gray graph behind the technology adoption lifecycle has an axis labeled “Market Share.” No, in no way is a technology firm allowed to capture 100% of the market share. The maximum is 74%. After that, you have a monopoly and your business is in violation of antitrust law. The EU is probably stricter than the US. That 74% is the US threshold.
  • In figures 1, 2, 3, and 5, the second phase is called the “Early Adopters.” Under Moore’s version, this phase is more accurately called the bowling alley. It is where we sell into the vertical markets by selling to one B2B early adopter in each vertical. We would enter six verticals with a product conceived by the early adopter. That product would be built on the technology we are trying to get adopted. Products are just the means of getting the underlying technology adopted. The product visualization is the early adopter’s alone. The idea is not ours. We sell to six early adopters. This takes time. There is no hurry. We have to ensure that each of these six early adopters achieves their intended business advantage.
  • The population percentages for each phase are accurate in figure 3.
  • In figure 4, the Chasm is correctly placed, but the early adopters are to the left, aka before the Chasm, and their vertical is to the right. It is not accurate to call the entire phase where the Chasm occurs the early adopters. There is a two-degrees-of-separation network between the early adopter and their vertical. Sales reps find no particular advantage in attempting to sell to a third degree of separation. Selling to that network constitutes the central issue of the Chasm.
  • Figure 4 also splits the early and late majorities in the wrong place.
  • In figure 5, the Chasm is incorrectly placed. The Early Majority is really the horizontal, usually the IT horizontal. The Tornado sits at the entrance of this phase, the horizontal, not the Chasm. The Chasm sits at the entrance of the verticals.

One of the problems that Moore encountered was the inability of managers to know where they were in the TALC. These figures do not agree with each other, so how would managers using different versions come to agree.

I’ve made my own changes to the TALC. First, the left convergence of the normal is well after the R&D, aka science and engineering research that firms no longer engage in. The left convergence is long after the research has gained bibliographic maturity. The left convergence only happens when researchers with Ph.D.’s and master’s degrees decide to innovate after having invented. They happen long before the TALC. This doesn’t look like how we innovate these days. These days we innovate in the late phases and innovate in a scientific and engineering-free idea-driven manner with design thinking innovating around the thinnest of ideas. These early phases, the phase before the late majority start with discontinuous innovation. These days in the phases after the early majority we innovate continuously. We don’t try to change the world. We are happy to fit in and replicate as directed by the advertising-driven VCs. The VCs demand exits so quickly that we couldn’t change the world if we wanted to.

The second change was in the placement of the technical enthusiasts. They are a layer below the entire TALC. They are the market in the IT horizontal. But, they work everywhere.

The third change involves integration with my software as media model. Each phase changes its role as a media. A media has a carrier and some carried content. All software involves the stuff used to model, and the content being modeled. Artists use pens, inks, paints, bushes, and paper. Developers use hardware, software, code, … Artists deliver a message. Developers deliver a message at times more obvious than at other times.

The fourth change is my labeling the laggards as the device market and the phobics as the cloud. I do this because these populations do not want their technology use to be obvious. The phobics use technology all the time, but with deniability. They use their car, not the computer that runs the car. Task sublimation and pragmatism organize the TALC. The phobics get peak task sublimation. This is where the technology disappears completely outside of the technical enthusiast population.

Here is a revised view of the TALC that incorporates my extensions and changes.

02 Revised TALC

The end is near. The underlying technologies disappear at the convergence on the right. Then, we will need new categories that we can only build from discontinuous innovation. If you don’t read the journals, you won’t see it coming. And, if you spent your life doing continuous innovation, you won’t be able to innovate discontinuously.

Another figure out on Google correlates Gartner’s Hype Cycle with the TALC. But, this Gartner Hypecycleone is absolutely wrong. Gartner has nothing to say about technologies in the vertical. Gartner starts with the IT horizontal. If the horizontal is not the IT horizontal, Gartner has nothing to do with the TALC. The Chasm happens a long time before the Trough of Disillusionment. The Hype Cycle starts at the tornado that sits at the entry into the IT horizontal.

 

I’ve made the necessary adjustment in the following figure. The Hype Cycle does Gartner Hypecycle and TALC Modifiedmanifest itself in the IT Horizontal and all subsequent phases. One Hype Cycle does not cross from one TALC phase to another. Each phase has its own hype cycle. I’ve only shown the hype cycle for the IT Horizontal.

The original figure was found in a Google image search. It was sourced from foundersresearch.com.

The reason I moved the Hypecycle is that in the search for clients in the vertical, IT is specifically omitted, and IT is not involved in the project. The client has to have enough pull to keep IT out. The clients would be managers of business units or functional units other than the eventual intended horizontal that you would enter in the next phase. The Chasm and the earlier adopter problems discussed relative to earlier graphics is apparent here.

The second tornado came up in Moore’s post web 1.0 work. It happens after a purchase but before integration. The VCs get their money on completion of the purchase. The acquiring company gets value from the M&A only after the integration attempt succeeds.  The AT&T acquisition of DirectTV had a very long tornado. That tornado is probably done by now. Most M&As fail. Many M&As are done solely to ensure the VCs recover their money. These are not done because the acquired company will generate a return for the acquirer. The underlying company fades into oblivion shortly after the acquisition. I’ve put both tornados in the next graphic. The timing of the M&A is independent of phase.

MA

In most figures, the acquiring company is shown moving upwards from the M&A. That is incorrect. The acquiring company is post-peak, post early majority and is in permanent decline. The best that can happen is that the convergence on the right will be moved further to the right granting the acquirer more time before the category dies. The green area in the figure reflects the gains from a successful integration, which happens to require a successful second tornado.

What was not shown was the relation of the first tornado to an IPO that pays a premium. That only happens with discontinuous innovation, and only in the early phases of the TALC. With the innovations we do these days, we are in the late phases of the TALC, so there is no premium on the IPO.  Facebook did not get a premium on their IPO.

One aspect of today’s TALC that I have not worked out is how the stack of the IT horizontal is cannibalized by the cloud.

Back when I gave my SlideShare presentation in Seattle in 2009, a lot of people didn’t feel that the TALC was relevant. It was still relevant then. It is still relevant now. We leave much money on the table by rushing, by being where everyone else is, by quoting the leaders of the early phases while we work in the late phases. We settle for cash, instead of the economic wealth garnered by changing the world. If we set out to change the world, the TALC is the way.

Enjoy.

 

 

 

 

Advertisements

Generative from Constraints, a Visualization

May 23, 2018

I came across a tweet from Antonio Gutierrez from geogeometry.com. Several constraints on a plane form a triangle. That triangle could have been a point before the constraints were loosened enough to give us some space within that triangle. More constraints would just give us a different polygon.

The loosened constraints required some room for continuous innovation. The point that became the triangle could be thought of as a “YET” opportunity of a problem that couldn’t be solved yet. But, with the triangle the opportunity awaits. So we dive in from some point of view where we can see the point at some distance. We establish a baseline from our the point of our view, the origin, to the center of the triangle. From that origin, we project three lines up to and beyond the triangle. This volume is code. At some point above the constraint plane, we take a slice through that volume of code, the blue triangle, Generative From Constraints over Time from Originand ship it. We continue to work outward. This would involve very little rework.

Alas, things change. The constraints contract (red arrows) causing us rework, or widen (green arrows) to give us space for new opportunities. The black triangle at the intersections of the constraints could widen or contract in parallel to our current boundaries (black arrows). Or, we could move our origin up or down to widen or narrow our current projection. That’s three classes of change. Each class gives us different volumes to fill.

In my game-theoretic illustrations, the release is always in a face-off with the requirements, such is the nature of design in the axiomatic sense of requirements from the carried content as assertions balanced against the enabling and disabling elements of the carrier technology. The projection doesn’t go hockey stick like into the constraints of the underlying geometry. There is always a constraint up there that’s much closer than we’d like to admit. Goldratt insists that there is always another constraint. And, in hyperbolic geometry, there is always a convergence at the nearby infinity.

In another view, the first line (red) from the origin through the center of the triangle and API - w Carrier and Carriedout into space is where we start the underlying technology. It grows outward thickening the line into a solid with the pink triangle as the base of the carrier technology. The carried content is built outward from the carrier core.

Constant change can be managed. Moving the origin down contracts the code volume. Moving G towards B contracts the code volume. Moving E towards A contracts the code volume. And, moving F towards C contracts the code. You can know before you code where rework is required and where your opportunities are to be found.

I’ve kept this simple. You can imagine that your carrier and your carried content have their own constraints, timeframes, and rates. There would be two planes, two centerlines, two triangular solids intersecting on the place representing what we will ship. We could slip in a plane to project onto and out from. Oh, well.

Enjoy.

 

Holes II

May 8, 2018

This week I revisited fractional calculus. A few months ago, someone on twitter tweeted a link to a book on fractional calculus. I didn’t get far. My computer crashed, so I lost my browser tabs. I didn’t reload them, because I had so many the browser was slowly doing its job, which apparently is collecting vast numbers of tabs of readme wannabes.

The topic came up again. I’m not sure the original link got me to the Chalkdust article, or if I had to Google it. The content was less complete, and not historical at all. But, you come away with two methods of getting the job done.

The article ended with a graphic that blew me away when I look at it from the perspective of discontinuous innovation. The discontinuity is large. It went on to hide, you might say, another discontinuity. I’m always asked what discontinuities are. I try never to make the mathematical answer to that question. The Wright brothers were not math equations.

Fractional CalculusSo here is the figure from the article. Do you see the discontinuities? The first one is glaring if you’re always looking for and needing discontinuities. Much like the discontinuities that the Mittag-Lefler Theorem, discussed in my last post, Holes,  lets us generate one or more discontinuities are essential to discontinuous innovation. There is profit in those holes. They are profit beyond the cash plays of continuous innovation, the profit of economic wealth that accumulates to the whole, the “we,” not just to the “me.” They are profit in the sense of new value chains, new careers, and revised ways to do jobs to be done.

Fractional Calculus - DiscontinuityI marked the figure up to uncover the discontinuities. We can start with the plane ABCD. The plane is outlined with a thin blue line containing the red surface from which the differentiation process departs. I drew some thick red lines to outline the hole where the process lifts the differentiation process above the plane.

There is a shadow that is visible through the front surface of the process. It was visible in the original graph. Highlighting it hides it. The thin orange lines highlight that surface.

D8 and D9 do not intersect. The third dimension lets them slide by each other without intersecting. When confronted with an intersection of constraints, look for a dimension that separates them, or look for a geometry that separates them. As product managers, we just have to look for the mathematicians and scientists that separate them. Product has always been about breaking or bending a constraint. Here we broke one. It looks like all we did was bend a constraint as of yet.

The hole is on the floor of the atrium, not on the canvas comprising the surface of the tent.  I drew a line parallel to the y-axis and put a hole on it so we could see the discontinuity. It’s not a hole that is a point. It is an area, an area on the plane. I drew a gray line across the plane to characterize the hole on that line. These scan lines don’t have to be parallel or orthogonal to the x-axis, but a polar or complex space would not simplify what we are doing here.

Everything under the surface of the graph and above the original plane is the hole. Another plane would characterize the hole differently.

That’s the first discontinuity.

Having read the article, I know that fractional derivatives involve deriving and then adding an approximation of the fractional component, or deriving past the integer power and subtracting the fractional component. In integer calculus, it’s all about functions until you get to a constant, a number. And, when you get a constant of zero, you’re done. There is a wall there. There is a hole on the other side of that wall into which no mathematics I know goes to take a swim. Yes, the differentials can be negative. We call that process integration. But, the switch between analysis and the approximation by the Gamma function is significant as is the switch between analysis and number theory.

I drew an axis above the graph in the sense of derivatives only omitting integration and projected the boundaries between equations, numbers, and zero. At zero, the zero deflects integration when zero is a number, rather than a function with the value of zero. It’s a gate. When that zero is the value of a function, integration passes unimpeded into the negative differential region.

Most of the time the “Does not exist” answer to the equation just means that we don’t know the math yet. Yes, we cannot divide by zero until calculus class, then we divide by zero all the time. The Mittag-Lefler theorem welcomes us to put holes where we need them. The mathematics is simpler without holes, so mathematicians sought to get rid of them. But, as product managers, we need our holes, if as product managers you are commercializing discontinuous innovation.

On our plane, point D at the far left where we’ve gone to number. The second hole is to the left of the orange line I projected up to our function-number axis. I don’t yet know what’s on the other side of line. Now, I’ll have to go there.

Enjoy!

Holes

April 27, 2018

I’m about discontinuous innovation. I’m asked at times to define discontinuities. Well, Kuhn’s crisis science is one answer. Mathematical holes is another. Anything on the other side of a constraint that nobody knows how to cross, yet another. Or, a logic that drives across an inconsistent space. Or, the line approaching the horizon of a hyperbolic space that never arrives because it goes to a limit and can’t pass that point of convergence. Or, the simpler case of anything not continuous.

As marketers, our answers are simpler. If we can bring it to an existing market from within an existing category, it is continuous. We’ll make some money, but we won’t change the world. We put our innovation into the late mainstreet, the device or the cloud market and starting there we leave a lot of money behind. Don’t worry, everyone else left that money behind as well.

But, if we have to take the long road of complete adoption in a nonexistent category and face the nascent bowling alley, the B2B non-IT early adopter, and the so-called non-existent Chasm. We don’t leave any money behind, and we have to create careers and a value chain, those being outcomes from generating a category which in turn generates larger financial returns than what our own intra-firm managerial accounting tells us, then the innovation is discontinuous.

I posted this after hearing an interview about the MittagLeffler’s theorem.  The Mittag-Lefler Theorem is about holes or more to the point how to make holes with a single function. Or, simpler, how to describe them. The holes exist. The function doesn’t. The point of the function is to process the holes to some end. This function can be described in a manner that it deals with all the holes, not just one. Green’s theorem deals with one hole you encounter while doing the integration.

The holes of the Mittag-Lefler Theorem show up in complex analysis. Kicking it back to marketing, we seek one hole, just one, but this theorem tells that there would be many holes. That’s the point of the bowling alley in Moore’s technology adoption lifecycle. We put the technology into a product built for one early adopter in their vertical market. This being one lane of our bowling alley. This being one hole. Then, when we have the capacity, we put the technology in another product for another early adopter in another vertical market. This being another lane of our bowling alley. This being another hole. We have to do this a total of six times across seven years. Discontinuous innovation is not fast. There is an eventual point, success in the tornado we face as we enter the horizontal maybe ten years later. But there is a point.

Those six lanes would fill six holes with six different value propositions in six different vertical markets, but the same underlying technology. Each of those client engagements would be in a different place at different heights in the industrial classification tree. That puts the holes at different heights from the complex plane.

But, back to the math. One kind of function that requires the theorem are Meromorphic functions like the one below. The cool thing is that you can write a function that Holes 01describes all those holes. This is a relatively simple function. The holes could be all over the plane, and still, a single function would handle them. I can imagine using a Fourier sum to get this done. A Fourier analysis would give us a collection of trig functions describing a wave that hits these holes. That sum would be a sum of different waves where each wave hits some of the holes.

 

This example is simple. It only requires one frequency. The cosines go to zero at each hole. We take the reciprocal of that, aka we divide by zero, and a hole results. These would be more involved because the holes are of different sizes. I don’t know how to do that yet. But, this is a start. This is a good mystery if your math isn’t there yet.

Holes 02

I drew the red line to say where the wave would be oscillating. The thick black line is the wave, a cosine wave. The reciprocal of the cosine gives us the hole. Those holes are where we will be making our money for a few years, seven years plus development time and value proposition development and execution, plus two years or more in that vertical. This if you’re in the last of the six lanes of your bowling alley.

The function f(x) is not a complex function on the complex plane, but if the red line has an angle of zero degrees, the function is complex. The origin is at (0,0).

Fourier analysis can break down a much more complicated signal into a wider set of waves, and sum all of those waves into a single function. I’ll add a few more complications to the figure.

Holes 03So this was my first try. It’s wrong. This is complex analysis, so the waves are on an axis through (0,0) and would have a different complex variable multiplying each wave. Regardless of where you put the holes a sum of complex trig functions can get us there. The figure shows the component waves that a Fourier analysis would deliver.

Holes 04

Here, I have put a hole out there in complex trig. I should have drawn the black ellipse centered at the origin. This polar complex view is far simpler than the waves shown in the previous figure. There might be more waves to add up here, but it is clearer. I’m not sure the trig functions are correct, but this is my best and last attempt for now.

I raised it for some clarity, but that puts the height in the equations. I deliberately drew the height of the new hole some depth w below the whole, so for this wave the height adds v and subtracts w. The reason I put the height in the equation will take us back to the marketing. Back to a vertical issue relative to where we enter the vertical market associated with the hole.

Verticals are organized by the industrial classification tree. Every vertical is a subtree of the classification tree. Don’t enter the vertical at the top of the subtree, nor at the bottom of the subtree. Try to leave yourself some room to generalize towards management at the root of the subtree or to specialize towards the detailed work in the leaves of the subtree. The most difficult work would be to implement and sell to siblings.  There will be enough to do for the early adopter client and their company.

The height of the hole,  w, would match with the vertical height of the client’s business in the industrial classification tree.

We will look at the vertical in the technology adoption lifecycle (TALC). The vertical is just one of several normals that are summed into the TALC. They have not been drawn to scale. Keep in mind that the device/laggard and cloud/phobic markets are small and short in terms of time.

TALC Hole

The hole is shown in the top layer of the figure showing individual normals that get summed into the TALC shown in the bottom layer of the figure. The hole is on the far right. The normal for the vertical would replace the Meromorphic function we used in the previous figures. The hole is associated with a single lane in the bowling alley.

There would be six lanes for a given discontinuous innovation. They would be entered into successively until the company could afford and is staffed to do more projects at once. One early adopter engagement, particularly the early ones for a given technology, would take two or more years. That these engagements are stretched out over time, satisfies the requirements of the Mittag-Lefler theorem that insists on the holes being clearly separated.

Now, we’ll fill the bowling alley.

Bowling Alley

I’ve used a fragment of the industrial classification tree to find a B2B early adopter in the middle of their vertical. Then, I measured the depth of the early adopter business in that tree to the total depth in the classification tree. Then, I put the hole for their position on the normal of the aggregate TALC. All six verticals were measured in the same way and placed on the aggregate TALC. Then, I used the polar form to build the hole accessing functions. There are six verticals, one for each early adopter. We need six different verticals, rather than six engagements in one vertical. I then set up a rough schedule for getting those six applications of the underlying technology done.

Once all six verticals have been built, we ensure that the early adopter’s value expectations are met. Then, we help them write their business case. We will use that business case when we market and sell to the early adopter’s network through the first three degrees of separation.

Once we have built successful applications in those six verticals, we can sell the underlying technology more directly into the IT horizontal. It takes quite a while. It is not the flash in the pan miracles we see in the consumer phase. Time is money, earned money.

Enjoy.

 

 

 

 

“Pinpoint; How GPS is changing technology, culture, and our minds”

April 25, 2018

In Greg Milner’s “Pinpoint, ” James Cook, a British sea captain in the mid- to late-1700’s sought to discover how the Polynesian’s navigated. The Polynesian navigators could demonstrate their abilities, but they could not say what they knew. The knowledge Cook sought was implicit.

By the end of the book, the knowledge was a collection of memories that were not to be forgotten. This said some 200 years later. Consider the agilists shipping errors for 200 years.

Consider what spellcheck does. It reduces our confidence in our ability to spell. It attacks us.

GPS reduces our ability to know where we are in the physical setting. GPS is more of a clock than a compass. I don’t wear a watch anymore. I don’t care what the bus schedule says. I just want to know how far I am from the next bus. Sometimes, the buses around here just never show up. One day, with a traffic jam downtown, three buses in a row never arrived.

GPS reduces our memories of the contexts of places. Places become numbers, numbers in a particular context that have nothing to do with specific places. The developers of GPS picked a representation and evolved that representation. When they solved the navigation problem, they extended the representation because they could see things that they never imagined. They can determine the humidity levels in the air. They could sense the movement of land masses. They can take over from the seismographs once the seismographs get swamped. They could notice when the Earth’s center of gravity changed. They know when masses of water move.

But, mostly, they change our memories of place.

We live in an age that would rather disregard the experts. We deliver products for the novices. We don’t ask the experts of what their cognitive models consist. We deliver software at the level of the introductory class.

A tweet this week linked to an article on how chaos researchers are ignoring formulas and just looking at the data, at the trajectories themselves using machine learning. Their system works. This hints at developing software where you don’t talk to the users or the customers. You just look at the data. The trap will move from explication to illumination. Many sensors need illuminators that make the sensed visible before they can capture the data. This is all well and good, but neural nets can’t tell us the equations, the conceptual frameworks. They capture the results from a given dataset. But, sensing the differences between salamaders and lizards, it can’t tell you what those differences are.

When we code, we ask questions about a situation that differs by one bit. If they differed by a kilobyte, it would be much easier to tell them apart. We could get by with much less data. When the difference is a mere 1 bit, we need upwards of 600k examples. That’s big data.

“Pinpoint” was an interesting story of adoption and adaption, competition and collaboration–coopertition. Product managers should find it a good read.

 

A Few Notes

March 20, 2018

Three topics came up this week. I have another statistics post ready to go, but it can wait a day or two.

Immediacy and Longevity

I crossed paths with a blog post, “Content Shelf-life: Impressions, Immediacy, and Longevity,” on Twitter this week. In it, the author talks about the need for a timeframe that is deals with the rapid immediacy and the longevity of a product.

When validating the Agile-developed feature or use case, achieving that validity tells us nothing about the feature or use case in its longevity. When we build a feature or use case, we move as fast as we can. The data is Poisson. From that, we estimate the normal. Then, we finally achieve a normal. Operating on datasets, instead of time series hides this immediacy. Once that normal is achieved, we engage in statistical inference while at the same time continuing to collect data to reach the longevity. This data collection might invalidate our previous inferences. We have to keep our inferences on a short leash until we achieve a high sigma normal where it is big enough to stop moving around or shrinking the radius of our normal.

In the geometry sense, we start in the hyperbolic, move shortly to the Euclidean, and move permanently into the spherical. The strategies change, not the user experience. The user population grows. We reach the longevity. More happens, so more affects our architectural needs. Scale chasms happen.

The feature in its longevity might move the application and the experience of that application to someplace new, distant from the experience we created back when we needed validity yesterday, distant from the immediacy. The lengthening of tweets is just one example. My tweet stream has gotten shorter. That shortness makes Twitter more efficient, but less engaging. I’m not writing so many tweets to get my point across. There is less to engage with.

This longer-term experience in the is surprisingly very different. In the immediacy, we didn’t have the data to test this longest time validity. Maybe we can Monte Carlo that data. But, how would we prevent ourselves from generating more of that immediacy data in bulk that won’t reflect the application’s travel across the pragmatism gradient?

The lengthening of the tweets probably saved them some money because they didn’t have to scale up the number of tweets they handled. They take up more storage, but no more overhead, a nice thing if you can do it.

Longest-Shortest Time

Once the above tweet took me to the above post on the Heinz marketing site, I came across the article, “The Longest Shortest Time”  there. The daily crises make a day long, but the days disappear rapidly in retrospect. The now, the immediacy is hyperbolic. The fist of a character in a cartoon is larger due to foreshortening. Everything unknown looks big when we don’t have any data. But, once we know, we look back. Everything is known in retrospect. Everything is small in retrospect. Everything was fast. That foreshortened view was fleeting. The underlying geometry shifted from hyperbolic to Euclidean as we amassed data and continues to shift until it is spherical. The options were less than one, then one, then many.

Value in the business sense is created through use. Value is projected through the application over time into the future from the past, from the moment of installation. That future might be long beyond the deinstall. The time between install and deinstall was long but gets compressed in retrospect. The value explodes across that time, the longest time. Then the value erodes.

In the even longer time all becomes, but a lesson, a memory, a future.

Chasm Chatter

This week there were two tweets about how the Chasm doesn’t exist. My usual response to chasm mentions is just to remind people that today’s innovations are continuous, so they face no Chasm in the technology adoption lifecycle (TALC) sense. They may face scale chasm during upmarket or downmarket moves. But, there are no Chasms to be seen in the late phases of the TALC, the phases where we do business these days.

Moore’s TALC tells us about the birth and death of categories. Anything done with a product in an existing category is continuous. In this situation, the goal is to extend the life of the category by any means, innovation being just one of the many means. VCs don’t put much money here. VCs don’t provide much guidance here. And, VCs don’t put much time here either. The time to acquisition is shrinking. Time to acquisition is also known as the time to exit. In the early phases, all of that was different.

Category birth is about the innovator and those within three degrees of separation from the innovator. That three degrees of separation is the Chasm. It’s about personal selling. It’s not about mass markets. It’s about a subculture in the epistemic cultural sense. It’s a few people in the vertical, a subset of an eventual normal. It’s about a series of Poisson games. It’s about the carried content. The technology is underneath it all, but no argument is made for the technology. It isn’t mentioned. The technical enthusiasts in the vertical know the technology, but the technology explosion, the focus on carrier is in the future. It is at least two years away and as much time will pass as needed. But, the bowling alley means it is at least seven years away.

Then comes, the early mainstreet/IT horizontal. The tornado happens at the entrance. Much has to happen here, but this is a mass-market play.

After the horizontals, the premium on IPOs disappears. We enter the late phases of the TALC where innovation becomes continuous and no new categories are birthed. This is the place where people make errant Chasm crossing claims. This is where all the people claiming there is no Chasm have spent their careers, so no, they never saw a Chasm. They made some cash plays. They were serial innovators with a few months on each innovation, rather than ten years on one innovation that did cross the Chasm. Their IPOs didn’t make them millionaires because there is no premium. The TALC is converging to its right tail. The category is disappearing. They cheer the handheld device, a short-lived thing, and they cheer the cloud, another even shorter-lived thing, the end of the category where the once celebrated technology becomes admin-free magic.

So yes, there is no Chasm. But, my fear is that we will forget that there is a Chasm once we stop zero-summing the profits from globalism and have to start creating categories again to get people back to work. Then, we will see the Chasm again. It won’t be long before the Chasm is back.

Enjoy.

 

 

 

Nominals II

March 15, 2018

I left a few points out of my last post, Nominals. In that post, the right-most distribution presents me with a line, rather than a point, when I looked for the inflection point between the concave-down and concave up sections of the curve on the right side of the normal distribution.

A few days after publishing that blog post, it struck me that the ambiguity of that line had a quick solution tied to the fact that the distance between the mean and that inflection point is one standard deviation. All I had to do was drop the mean from the local maximum at the peak of the nominal and then trisect the distance between that mean and the distribution’s point of convergence on the right side of that nominal’s normal distribution.

Backing out of that slightly, every curve has at least one local maxima and at least one local minima. A normal distribution is composed of two curves one to the right of the mean and another to the left. Each of those curves has a maxima and minima pair on each side of the mean. The maxima is shared by both sides of the mean. A normal that is not skewed is symmetric, so the inflection points are symmetric about the mean.

01 min max IP

Starting with the nominals comprising the original distribution, I labeled the local maxima, the peaks, and local max minima, the points of convergence with the x-axis. Then, I eyeballed each line between the maxima and minima pairs to find the inflection point between each pair. Then, I drew a horizontal line to the inflection point on the other side of the normal. Notice the skewed normal is asymmetric, so the line joining the inflection points is not horizontal. Next, I drew a vertical line down from the maxima of the normal distribution on the right. Then, I divided the horizontal distance from the maxima to the minima on the right into three sigmas or standard deviations. The first standard deviation enabled us to disambiguate the inflection point on the right side of the distribution.

The standard normal is typically divided into six standard deviations–three to each side.

02 IP

Here I’ve shown the original distribution with the rightmost nominal highlighted. The straight line on the right and the straight line on the left leaves us unable to determine where the inflection point should be. My guess was at point A. The curvature circles of the tails did not provide any clarity.

I used the division method that I learned from a book on nomography. I drew the line below the x-axis and laid out three unit measures. Then, I drew a line from the mean and the x-axis beyond the left side of the first unit measure. Next, I drew a line from the distribution’s point of convergence on the right side of the normal beyond the right side of the third unit measure. The two lines intersect at point 3. The rest of the lines are projected from point 3 through the line where we laid out the three unit measures. These lines will pass through the points defining the unit measures. These lines are projected t the x-axis.

Where the lines we drew intersect with the x-axis, we draw vertical lines. The vertical line through the mean or local maxima is the zeroth standard deviation. The next vertical line to the right of the mean is the first standard deviation. The standard deviation is the unit measure of the normal distribution. The vertical lines at the zeroth and first standard deviation define the width of the standard deviation. The vertical line demarking the first standard deviation crosses the curve of the normal distribution at the inflection point we were seeking. The point B is the inflection point. We found the standard deviation of the rightmost normal without doing the math.

I put a standard normal under the rightmost normal to give us a hint at how far our distribution is from the standard normal. At that height, our normal would have been narrower. The points of convergence of our normal limit the scaling of the standard normal. A larger standard deviation would have had tails outside our normal.

03 Added Standard Normals

Here I’ve shown the six standard deviations of the standard normal. I also rescaled standard normals to show how a dataset with fewer data items would be taller and narrower, and how a dataset with more data items would be shorter and wider. The standard normal with fewer data elements could be scaled to better fit our normal distribution.

In the original post, I wondered what all the topological torii would have looked like. I answered that question with this diagram.

03 Torii

Enjoy.

 

 

Nominals

February 25, 2018

A tweet sent me to “Mean, Median, and Skew: Correcting a Textbook Rule.” The textbook rules are about the mean being in the long tail and the mode being in the short tail. The author discussed exceptions to this rule. Figure three presented me with a distribution that the author claims to be a distribution that was an exception to the textbook rules. The author claims the distribution is a binomial. I annotated the figure. It’s definitely some kind of a nomial, but looking closer, it is not a binomial.

00 Original Alleged Binomial

The nominal on the right side of the distribution shows us what we see if we look at the side of any normal. An aggregate curve comprised of a concave downward curve and a concave upward curve with an inflection point between them, a single inflection point between them.

The distribution on the left side is not the result of a single nominal. There are many inflection points. The left side of the distribution is concave down, concave up, concave down, and concave up. We can say the left tail is single tail comprised of two presented lines, or we can say they are the overlap of two different distributions. That second concave down hides a distribution inside the base distribution.

The distribution gets called a binomial because it has two prominent peaks. But the left peak is an aggregate of at least one more nomial. Otherwise, we would add another set of inflection points. When making an argument about where the mean, median, and mode are we have to consider each nomial to have its own triple. So there should be at least two triples, rather than one, as shown in the figure. I called the triple we were presented with an error, but it does present us with one of the exceptions the author wants to talk about. From this, we can take away the idea that these aggregate statistics hide more than they inform. I found myself in a Quora discussion on separating the underlying distributions of a binomial. There is math for that, math I do not know yet.

I am working on the assumption that all the underlying distributions are normal, a base assumption that is routinely made in statistics.

The graph hides much as well so I drew what I expected the distributions under the given “binomial” would be. I just eyeballed it.

01 More Nomials

I used arrows that match the color of the curve to show the concavity. Extra probability mass shows up at the intersections where distributions meet. I’ve labeled the probability mass at the intersections as gaps. Given the underlying distributions are only approximations, I didn’t make the green distribution, distribution 1, fit perfectly, so the thin layer of the second gap from the beginning lays on top of the distribution without involving a distribution. I used three different distributions to account for the tail convergence on the right. This gave rise to a gap. I didn’t catch this when I drew the figure. As I write this, there is no gap there. The red distribution accounts for that probability mass.

I went with a skewed distribution, distribution 1, to account for the second concave down section of the curve on the left side of the “second” nominal. A normal wouldn’t bulge outward under the exterior nominal, the black normal. A skewed normal has a long tail and a short tail. The intrinsic curvature of any long tail is low, so it has a large radius. The intrinsic curvature of any short tail is high giving us a small radius. The mean of this distribution is to the left. The median pushes the mean and mode apart symmetrically about the median. The median for distribution 1 leans to the right.

I went with three peaks on the left side of the “binomial.” I did this because distributions 2 and 4 have different heights. I know of no rules that would drive this decision. They could easily be one distribution.

The rest of our “binomial,” actually as demonstrated, it is a multinomial instead. We’ve ended up with five distributions so we would have five different triples of mean, median, and mode. These triples were aggregated in the author’s numeric results. We can take it that when the mean, median, and mode are the same, we have a standard normal. The textbook rules about the tails and their relationships to the mean and mode still stand. Otherwise, we have numbers generated from an aggregate normal.

Don’t just accept the “binomial” allegation. If the numbers don’t make sense, they don’t make sense. When numbers don’t make sense, you’ve got more sense to make.

As a product manager, I don’t want to aggregate and drive that into a product that fits no one.

I went on to play with the “binomial” distribution some more.

I started with vertical slices for the Riemann integral. I also did this to give me a hint towards the factors involved in each slice. Due to my use of raster graphics, some slice lines are thick, because the intersections of the distributions are not points. Some intersections are lines. The point intersections give rise to vertical lines. The line intersections give rise to rectangles. Each vertical slice in those rectangles can differ. They are not uniform. Individual slices would still look like a solid rectangle.

02 Some Vertical Slices

The vertical lines tell us that at that moment in time, our organization if we worked at the underlying granularity, would represent some management adjustment to serve the underlying populations appropriately. This both the gray and light blue lines or rectangles.

The blue lines show us where the associated distribution converges with the horizontal axis. That horizontal axis would move relative to any upmarket or downmarket moves the organization was undertaking over a period of time. I labeled these as ordering changes. But, the gray lines are ordering changes as well. Orderings come up when computing binomial probabilities and in game theory.

The pink area shows the expanse of a single factor mixture. Part of that area shows the factor associated with the black distribution quickly slowing down. I labeled that part of the black curve “Fast.” And, it shows the factor’s deceleration showing. That labeled “Slow.” Otherwise, this slice is relatively stable. Note growth is not a positive notion here. In fact, the late phases of the technology adoption lifecycle, the orthodox management phase is post growth and in decline–constant decline. The only options are to focus, an upmarket move, or to drop the price and move downmarket. Neither guarantee growth in themselves.

From the mean of distribution 5, the purple distribution, All factors are in decline. But in the pink area, the factors are organized by a single constant factor curve.

In Upton’s “Aesthetics of Play”, the pink zone is a single play space. In his book, rules generate spaces and those spaces dictate process and policy. The technology adoption lifecycle(TALC) is based on this idea, but it is based on populations organized by that population’s pragmatism. The business facing that play space or population must eliminate its process and policy impedances to succeed. Addressed impedances constitute your organization’s design.

These spaces make those nascent moments when we don’t have a normal part of the difficulty with bringing another discontinuous innovation to market while sitting in the space where the category the company is in is dying. The pink space is that end-of-life space. Notice how different the pink space is to any slice on the left side of the aggregate distribution.

Upmarket and downmarket moves move the feet of the distributions, the points of convergence with the horizontal. The new space might have additional intersections of the nominal distributions. Where this is the case, the factors for the new slices would change. This would repartition the existing populations as well. Where the nominals are normal, the additional populations gained by the move would not change the nominals other than at the feet. In upmarket moves, keep them large enough to maintain normality, or expect exposure to kurtosis risk.

In our diagrams, the red distribution seems high, which implies that it needs more density. The number of data points needs to be increased. This also implies that there should be some skew, but it is not apparent. As a distribution gains probability mass, it becomes lower and wider.

When looking for inflection points, those points can be lines. The nominal on the right exhibit that behavior. I went looking for what that means mathematically. The inflection point is ambiguous. I crossed paths with symplectic geometry. They deal with the same problem. The nice thing businesswise about this ambiguity is that it grants you some time to switch from growth to decline or from fast to slow. The underlying processes of the business need to change at all inflection points. The deal here between a point and a line is that a point is a sudden change requiring proactivity, and a line requires less proactivity.

Then, I wanted to see the toruses involved. So I started with the normal distribution on the right side of the “binomial.” I used the original distribution, not the teased out distribution, so the distribution on the left only exposed its left side. fitting a circle to the curve on the left was less clear.

03 Curvature

Imagine if a tori pair was shown for each of the five distributions. Where a tori pair does not have the same radius in each constituent circle, there would be kurtosis, a pair of tails, and a median lean. The radii of the circles in that pair would change as the 2D slicings were rotated around the underlying distribution. The median lean results from the particular dimensions of the 2D slice. This generates some ambiguity in the peak, as the median for each slice would differ. By slicings, I mean taking slices around the circle giving us a collection of different slices. I do not mean rotating the same slice.

Where a tori pair had the same radius, the distribution has achieved normality. The kurtosis would be near zero, the median would no longer lean, and the mean, median, and mode would converge to the same value. The radii of the circles would not change as the 2D slicings were rotated.

Next, I took horizontal slices as in Lebesgue integrals.

04 Some Horizontal Slices

As discussed in regards to the vertical slicing, the gray lines indicate point intersections. The thicker gray lines indicate line intersections.

Where the vertical slice figure showed gaps, those gaps are comprised of a collection of Poisson distributions and a single collective normal. Poisson distributions come to approximate the normal when it has 20 or more data points. The normal is achieved without approximation when 36 data points have been collected. Breaking a normal into subsets can give rise to Poisson distributions. So there is risk involved with these considerations. I highlighted these with yellow rectangles around the labels.

The skewed distribution, the green distribution, has been highlighted with the same yellow as the Poisson distributions because having not yet achieved normality, much will change and those changes will be rapid as normality is achieved.

The red arrows show the direction in which I expect the distribution to change. The left arrow associated with the skewed distribution is only considering the movement of the foot, everything will change with the skewed distribution. The base “binomial” will most likely change and give rise to an apparent 3rd nominal on the exterior of the aggregate distribution. The down arrows associated with the peaks can be expected to lose height or amplitude as more data is collected.

The median of the skew would become orthogonal. The change in its theta is not indicated on the diagram.

The intersections of the distributions will change, so they are highlighted in yellow as well.

The factor analyses also change when looked at from a horizontal slice point of view. You can consider the factors across a horizontal slicing to differ from the factors across a vertical slicing. There would be a collection of cubes if both slices where made. Those cubes would be N-dimensional, but given our slicings would be 2D, it would get messy. cubing based on a factor analysis would be easier to operationalize in the sense of organizational design.

I labeled the slices. I had intended to provide a factor analysis for each slice. If I had the underlying data that would have been possible, but a graphical approach proved frustrating.

Next, I generated the probability of a portion of the AI slice under distribution 5, the purple distribution. A Lebesgue integral would achieve the same result.

05 Probability of a Portion of a Slice

The blue rectangle represents the probability mass under the purple distribution between the vertical constraints of the gray lines delineating that dimension of the slice AI.

The author went on to give several examples of other aggregate distributions. He used these distributions to explore how the mean, median, and mode violate our expectations. So the textbook rules are violated by aggregates of underlying distributions, multiple distributions. This is true of the “binomial” example. As a rule, only consider those statistics to be valid at the level of the constituent nomials, rather than the aggregate nominal. Aggregate nominals frustrate the expected orderings of the statistical tuples.

06 Mean Mode Median

I take it that the thick black line is the mode. On the left, we get the textbook ordering. Then, in the yellow rectangle to the right of 0.5, it changes to an exceptional ordering. At some point, it changes back to textbook ordering. And to the right of 0.75, the mean changes its tail association to being associated with the short tail. In the textbook ordering the mean is in the long tail. This is where using a single number for kurtosis does not make sense. It only made sense in the standard normal sense where the tails have identical values on both sides on the 2D slice involved.

The author went on to construct a distribution associated with the graph showing the tuple ordering exceptions. In a skewed normal, the median leans over to sit on top of the mode. This is the case in the aggregate distribution used here. The ordering is not exceptional, but the lean is not at the value of mode but along it. Where I annotated this as exceptional, the exception is the distance from the median to the mode. The ordering is not exceptional. It does, however, change the width of the separation between the median and the mode. The ordering is not symmetric around the median. The red lines are intended to show the median leaning on the mean so that the asymmetry relative to the mean, median, and mode is clear.

07 Exception Constructed Distribution

Then, I went on to explore the logic of the 2D slice. Here we are talking about the logic of the carried data, not the logic of the statistical carrier. The logic of the statistical carrier would be that of a normal distribution. With all the mathematical approximation formulas allowing us to convert from one distribution to another, we might ignore the logical constraints. I’m calling these distribution-to-distribution logical constraints the logic of the statistical carrier. The aggregation rules for a normal is an example of such carrier constraints. The carried logic is that of the collected data, rather than the collection and analysis of such data.

Logical consistency is tricky. Decades ago consistency was a true or false question. Was it consistent from the top to the bottom across every branch of the argument? These days that’s called absolute consistency. But now, we have relative consistency. It works from some absolute consistency to a branch of the argument that is consistent with itself and that base absolute consistency. Other branches would arise. Those branches would not demonstrate absolute consistency with other branches. This kind of consistency is relative consistency.

Statistically, the relative consistency would be a characteristic of each tail. Absolute consistency would be a characteristic of the core.

Relative consistency leaves us in a non-Euclidean space. That space typically would be hyperbolic involving manifolds, rather than functions. This calls into question the management practice of alignment and organizational structure.

08 Slice of Distribution and Logic

In this figure, the logic of the tails is highlighted in pink. The question marks indicate where one would define shoulders, outliers, and distant outliers.  What are your definitions of those boundaries? This is a 2D slice. Another 2D slice through the mean might require different decisions. Another slice would have a different set of curves. One of the slices would appear to be a standard normal with equal tails on both sides of its mean.

Relative consistency would start at the shoulder of a particular tail. Where you don’t differentiate the shoulders from the tails, a relative consistency starts with a particular tail. Each tail would have its own logic.

The last figure demonstrates the slices concept. The red line is closer to a standard distribution and its tails. The blue slice is definitely skewed. The thin blue line in the core is there to hint at the lean involved in that 2D slice. The red slice does not exhibit any lean. As more data of the dimension underlying the blue baseline is collected, the lean will disappear as will the asymmetry of the tails.

09 Slices Lean and Tails

As a manager, big data is great if you have large existing populations and large existing collections of relevant data. Continuous innovation thrives in this situation. But, do be cautious of Poisson scale subsets. And, be cautious of any distribution summed to the existing normals. That data might be Poisson. And, that distribution would be skewed and kurtotic bringing you their relevant risks. Discontinuous innovation is blank space inventions tied to an absence of any relevant populations. These innovations have tiny networks. Data collected from those networks will be small data, Poisson, pre-normal, and will move across the terrain. It will be a long time before it settles down, but at the same time, it is a long way from being a commodity, or something that orthodox management practice can handle. It is a long way from the spherical geometry of that orthodoxy. It is a long way from the Euclidean of LP2. It is hyperbolic. All that distance implies there is real economic wealth to be created, and there is plenty of time to capture it.

The data collection and relevant distributions will mature.

Snapshot statistics is not all that informative. What your distributions dynamically.

Enjoy.

 

 

 

Box-Whisker Charts

February 12, 2018

Twitter presented me with this box-whisker chart about perceptions of probabilities. The probabilities run from the most certain to the least certain. All these probabilities could 00bbe summed into a single normal distribution. I tried to put the footprints of all the distributions into a single footprint. I don’t have the tools.

Most of the distributions are skewed, so they are ellipses.

Each of these distributions appears to be mutual exclusive.

I already knew boxplots. So I tried to grasp the shape of the 00fdistributions. I annotated the above figure as shown on the right. I hacked the notation. I’ll discuss it in detail later in this post.

There are normal (N) and skewed (SK) distributions. Each of these box charts has a pair of tails, but as I went along, I realized there are three pairs of tails. Each pair of tails consists of a long tail (L) and a short tail (S). Once I realized there were three tails, I used L1, L2, L3, S1, S2, and S3 to label the tails. After that, I found tail pairs that were missing a tail. I used 0 (zero) to annotate them. Later, I realized that they are really don’t cares. Their lengths are unknown.

The outliers are annotated with red “if”s. Including outliers or excluding them should be a matter of established policy. The costs of writing code for a transient population of outliers can be quite, and needlessly, expensive.

I read What a Boxplot Shape Reveals About a Statistical Data Set and found a surprise. Boxplots assume a monomial distribution. The article compares two distributions with the same boxplot. 00eThey use the histograms to illustrate this problem. I’ve added the red text and the data point counts.

The distributions shown do not have enough data points to use the Poisson distribution to estimate the normals. The distributions have not yet tended to the normal, so they are skewed. The box-whisker chart would tell us more about the skew.

As I wrote this post, I looked back at the article 00dthat contained the first graph in this post. The article contained two graphs of the actual distributions summarized in those first two box-whisker charts.

In this figure, I labeled the outliers, They appear as their own distribution. I’ve also labeled the nomials.

00g The same labels apply to this figure, the second figure illustrating the distributions.

The next thing to look at is the normals being added together to give us those multinomial and binomial distributions. I have edited the figure to the right. I used the tails that I could see to provide the missing tail, the tail under the adjacent normal. Once all the tails have been provided, there is left over probability mass that appears where the two normals intersect. I colored those blue and called them “mix” as this is where mixture effects occur.

Later in the upper part of the figure, I just used red Bezier curves to suggest normals. Initially, I understated the number of nominals involved. Then, I found more than one inflection point on a given tail. These bulge out at the side of the distribution. These bulges are caused by another normal inside or under the covering normal. These can oscillate in some situation. But, the peak of the normal under is never exposed so you wouldn’t call it a nominal.

The previous figure shows us what the statistical distributions associated with the technology adoption lifecycle (TALC) would look like. They would be a series of distributions. They would not be a single distribution that just grows. The previous figure as looks like the pragmatism slices that comprise the TALC. Each pragmatism slice would have its own distribution. These distributions would aggregate into the TALC phase distributions.

While I was researching this, I watched a video on calculating multinomial probabilities. I watched the subsequent videos on this topic. It struck me that given independent, mutually exclusive probabilities used in these calculations gives rise to a histogram, which in turn takes us back to the box-whisker chart and individual distributions. It also takes us back to the finite probabilities of the long tail of feature use. Once you have stable frequencies in your long tail, you would have a set of probabilities that add up to one. Changes to the UI would change the frequencies and subsequently change the probabilities.

The figure above does not give us any hints as to skew or kurtosis. The box-whisker chart can provide some information. In the earlier histograms, the one on the right shows that we have a binomial distribution. The data sets for those distributions have too few data points so those distributions would be skewed. The peaks are medians. Those medians lean. They are not perpendicular to the x-axis. The lean pushes the mean and mode apart. With a few statistics beyond what the box-whisker chart is telling you, you will be able to determine how many nomials are involved.

Analysing a Box-Whisker Chart

012

We start with a box-whisker chart for a normal distribution.

 

011

Then, we examine the symmetries with two tests: the core test (A) and the tail test (B). Before confirming normality via the core test, the red line, the median, would be black.

To do the core test, we draw 45-degree lines from the cross both boxes from the shared location where the median intersects a rectangle containing the two squares as shown If the lines intersect the opposite corners the boxes are squares. This implies that the boxes are the same size and that the distribution represented by the box-whisker chart is symmetric in terms of the core of the distribution. If the diagonal lines intersect the sides at the same height, again, the distribution is symmetric.

Next, we do the tail test. We measure both tails to determine if they are equal, or shorter or longer lengths otherwise.

If the box-whisker chart passes both tests the distributed represented by the chart is symmetric, which in turn tells us that the distribution is normal (N). I annotate normal distributions with using a red capital N. I also show the median as a red line at an angle of 90-degrees. The median does not lean in symmetric normals.

I used tick marks to indicate that the whiskers are the same length as is done in geometry.

In this figure, we examine a box-whisker chart for a 02skewed normal. The boxes are not the same size. Doing the core test, we find that line for the left box intersects the box higher than the line for the right box. This demonstrates that the boxes are skewed. This was labeled with a red “SK.” Since we know the distribution is skewed, we can lean the median by taking the median’s cosign. This gives us the length of the median. As we angle this median, it will contact either the mode or the mean depending on which tail is long. Here we left the whiskers the same length. We labeled the long side (L) and the short side (S). I then drew the shape of the distribution in blue based on the information from the box-whisker chart alone.

Theta is the angle with which the median was leaned.

The mode and the mean are the same as the median in an unskewed normal. They separate symmetrically around the median in a skewed normal. They are shown, for this illustration only, as short, vertical black lines inside the box.

In box-whisker charts, the median is usually shown as a thicker line.

In this figure, we look for 03three pairs of symmetries. The distribution is normal, so the core and tail pairs in each pair are the same length. This will not necessarily be the case with skewed distribution.

I did not measure the outlier distances earlier. This is where that happens. if , where d is the distance function or metric, d(ab) = d(ac), d(ad) = d(ae), and d(af) = d(ag) then the distribution pairs are normal. Otherwise, the unequal pairs are skewed so they would have unequal core widths or tails.

Once we know what core widths or tails 04are long and which are short, we label them. Here the left core is narrower than the right core. All the other lengths are the same, but the asymmetry of the core makes all the tails on the left shorter in aggregate than those on the right. The summary notation of S and L were enough to convey all the relationships between the pairs of tails. The numbered notation gets more complicated later. Nothing guarantees a nice orderly set of relationships. Folding at the median will be informative in some cases.

In this figure, we get a messy ordering 05of the relationships. I’ve added some notation. Where you move from a short to long or long to short, the tails protrude. If everything on one side is short and everything on the other side is long protrusions are less likely. They are not impossible because of the relative nature of shorts and longs.

Swaps are fairly active things, so they constitute a sensitivity driving kurtosis risk.

I connected these swaps on one tail. The other tail is swapped as well. Here SW23 means there was a swap in the second pair of tails, and another swap in the third pair of tails. The cores are just the first pair of tails. SW23 just condenses SWAND SW3.

The next figure is a mess. Three 06measurements, members of each tail pair are missing. S1S2, and Sare missing. The thick line on the rectangle is the perpendicular median. The only whisker is to the right, so that is where the long tail goes, to the right. That means the median leans left. There are no outliers to the right, so S3 does not exist. I use zero to indicate non-existence.

Every outlier has been labeled with a red “if.” Every outlier causes us to consider whether to leave it in or take it out. The further away from the mean it is the more likely it will be eliminated. But standing business rules are better than ad-libbing here. Establish policies. Outliers are costly to serve.

In this figure, I have annotated the curvature 07of one of the pairs of tails. Given three pair of tails, there would be three toruses that could be generated by revolving the curvatures around the mean. A curvature is the reciprocal of a radius. This implies that high curvatures are tighter and smaller than low curvatures. The small orange circle has a tight curvature. The large orange circle has a looser curvature. A 2-D slice is shown. In n-D or 3-D, the two circles are part of the same torus revolving around the core of the normal. The surface would be smooth and continuous.

A tight curvature corresponds to a short tail as it is tangent to that tail. A loose curvature corresponds to a long tail as it is tangent to that long tail. As the distribution approaches normality, the curvatures equalize to some average curvature. The circles become the same size on both sides of the distribution in its 2-D view or slice. The curvatures of the standard normal are the same on both sides of the distribution.

Big data ultimately comes down to Markov chains that sequence individual distributions together. The original charts demonstrate how meaning is particular to place. Upton says as much in his The Aesthetic of Play, as did Moore’s Crossing of the Chasm.

Enjoy.

 

 

 

 

 

 

Skewed Normal

January 28, 2018

As a baseline, we’ll start with a top-down view of a normal distribution. The typical view is a side view. In the top-down view, the normal is the center of some concentric circles. In our graph, the concentric circles will have radiuses defined in terms of the statistical unit of measure, standard deviations. I’ve shown circles at 0,1,2, 3, and 5 standard distributions. The mean is shown at 0 standard distributions.

The core of the distribution is shown in orange. The horizontal view of the distribution defines the core as being between the inflection points (IP) of the normal curve. The core in a normal is the cylinder from the plane of the inflection points to the base of the distribution. The horizontal view can be rotated to align with the plane cutting the distribution for a particular dimension shown here as D1, D2, and D3. We only have three dimensions in this normal. With n dimensions, there would be n slices. With the normal, as long as the distribution is sliced through the mean, all the 2D projections would look the same. The normal is a symmetric distribution.

NormalWith a normal distribution, the mean, median, and mode have the same value. This and being symmetric is a property of the normal. More specifically, a non-skewed normal. A standard normal is not skewed.

The normal can be estimated with a Poisson distribution of at least twenty data points. The Poisson distribution will tend to the normal between twenty and thirty-six data points.

The normal is usually used in the snapshot dataset perspective, rather than in a time series sense. But, the time series sense is significant when you wonder if you’ve collected enough data. A dataset should be tested for normality. It is usually assumed. The tests for normality are weak.

Once the data achieve normality, the data tends to stay normal. The core and the outliers won’t move, and the standard deviation will stay the same. Until the data achieve normality, the distribution moves and resizes itself.

In the technology adoption lifecycle, the vertical phase will be the first time the normal is achieved. It will be a normal for the carried content. The horizontal, aka the early mainstreet market, has its own normal. The horizontal’s normal is for carrier components. The late market, likewise, has its own normal. In the late market, the focus is on carrier with the earlier carried discipline representing mass customization opportunities. The laggard and phobic phases are about form factors, carriers. Carried content may change is these phases. Carried content provides an opportunity to extend category life.

Preceding normality, the normal is skewed. In the next figure, I’ve put the skewed normal above the non-skewed normal.

Skewed Normal

Where the normal has a circular footprint, the skewed normal has an elliptical footprint. The median does not move. It tilts. This pushes the mode and the mean apart symmetrically around the median. The blue arrow shows how much the median tilts. The thick blue line shows the side view of the skewed normal. The core is shown in light orange. The tails are significant in the skewed normal. The skewed normal is asymmetrical. More on this later.

Each ellipse corresponds to the sigmas of our earlier diagram. But, the circular areas are the future. I’ve marked the outliers relative to the circular footprint of the non-skewed normal. The area I’m calling the deep outlier, the dark yellow population, is beyond what would be considered in the non-skewed normal. It would definitely be an error to collect data from that population, or since we sell to populations as we collect data from that population, it would be an error to sell to that segment of the population. Even after normality is achieved, outliers are more expensive than the revenues generated from that population.

The yellow populations are outliers, but they are outliers to the non-skewed normal. These outliers are shared by both distributions. The light green and even lighter green areas represent non-outlier populations that will be sold in the later normal, or as we sell to achieve normality. As the skewed normal achieves non-skewed normality, the ellipses will become circles. The edges located along the x-axis will move to the right. The tilted median will stand up vertically until it is perpendicular, and the mode and mean will converge to the median.

The ellipses would be thinner than shown. The probability mass under both distributions equals one so the ellipse would be less wide vertically than the circles. I had no idea about how wide those ellipses would be, but the figure is definitely wrong.

The skewed distribution exhibits kurtosis. I disagree with the idea that kurtosis has anything to do with peakedness. Other statisticians made this argument to me. The calculus view of the third moment disagrees as well. Kurtosis is about the tails and the shoulders as they relate to the cores. Some discussions ignore the shoulders. In this figure, I’ve included shoulders. I’ve used thick red lines and red text to highlight the components of the normal (N) and the skewed normal (SN). The normal only has one set of components. The skewed normal has two sets of components: one on the left, and another on the right.

I highlighted the shoulder of the normal. I highlighted the right and left shoulders of the skewed normal. And, lastly, I highlighted the right and left tails of the skewed normals.

The shoulders and tails are related to the cores. The normal core is a circle. The light orange ellipse of the skewed normal sits on top of it. I labeled both cores. The purple rectangle above the cores is the core of the skewed normal. The black core is the core of the non-skewed normal.

Kurtosis defines the curvature (κ) of the tails. I usually show these as circles defined as  κ=1/r. These circles are tangents to the tails of the normal. In a normal, these circles are the same size on for both tails. In a skewed normal the circles are vastly different in size. These circles in both cases generate a topological object: A torus for the normal, and a ring cyclide for the skewed normal. These topological objects are generated as we rotate 360 degrees around the median or mean of the normal. I showed this topological object in dark orange. In this figure, I showed them as ellipses. The circular version made the diagram very large. The ellipse for the ring cyclide on the left side is large. On the right, it is very small. This is due to the horizontal slice through the 3D objects. The xy-plane used to produce the slice through both objects. Both objects are smooth and continuous so another slice through the median would show a smaller circle on the left and a larger circle on the right. At some rotational angle, both circles would be the same, as in both curvatures would be equal. The thick vertical line through the median turns out to be the slice in which both curvatures would be the same. This curvature would be the average curvature.

When I put the left portion of the torus in the figure, the blue line representing the side-view of the normal was incorrectly drawn. The peak should have been at the mode. This was the second surprise. The median has more frequency, but it is tilted at an angle, an angle that makes it less high than the mode. The mode being the highest was one of those not yet know pieces of knowledge.

I’ll attempt a multimodal normal with opposing long tails. I was going to try to illustrate a such a normal. There can be a multiplicity of centrality tuples, skews and long tails. With the tools I used now, that would be a challenge.

I’m looking at the Cauchy distribution now. There is no convergence. But, Cauchy sequences converge based on ε. You can pick your convergences. A footprint would be zeros. Different values of ε would different footprints, and different conclusions of the underlying logical argument in the triangle model sense of the width and depth of a conclusion.

The first thing that surprised me in this post was how a portion of the outliers, the deep outliers, of the skewed normal is too far away from my market. And, how other portions of the outliers are outliers in both distributions. Another example of writing to think, rather than writing to communicate. Sorry about that.

Care must be taken to ensure this if you are going to market to outliers. I won’t.

Enjoy.