Social Analytics: Your Organization (and Society) as a Collective Intelligence

Originally published December 26, 2012

“Resistance is futile.”
Star Trek: Borg

If you ask anyone to describe the organization in which they work, chances are he or she will draw an organization chart for you. It will list business units, departments, people in charge, and it will look like a well-balanced hierarchy. This is how people are managed in the organization. Others, perhaps the more enlightened ones, will draw a value chain, structuring the various activities that the organization performs to add value. It will show how procurement and warehousing support production, and how production aligns with logistics and sales and marketing, all supported by HR, Finance and IT.  This is how work is being managed throughout the organization.

Complex Adaptive Systems

Both views are correct, but both views describe only a very limited aspect of what an organization really is. Boiled down to the essence, organizations are complex adaptive systems. They are dynamic networks of all kinds of interactions and relationships, and these relationships and interactions are continuously shifting based on tiny (or bigger) internal and external changes. As a whole, a complex adaptive system continuously adapts and learns. At best what you have is a fragile equilibrium. In most cases, the organization will operate far from these optimal conditions because of the energy required to manage the continuous change. This also explains why every organization is unique, even when it has the same strategy as another organization. External changes in the market trigger different changes within the system of the organization. Organizations evolve.

See if you recognize the following behaviors or patterns in your organization:
  • Sometimes it feels like decisions just emerge. No one can remember having said “yes” or “no” to a proposal, yet as a group we seem to go a certain direction.

  • The moment the competition makes a move, you huddle to see how to respond. You know from your contacts working for the competition that they huddle when they hear you huddle in an attempt to figure out what you huddle about.

  • You may have heard of “high-performing organizations,” but your organization seems far away from it. Keeping up with change, let alone driving it, is quite an effort already.

  • The best strategies and ideas you can remember came from groups of people with very diverse professional backgrounds. One of those moments of magic happened when sales, marketing, IT and legal evenly contributed.

  • It’s not what you know about the business, but who you know in the business.

  • When you boil it down to the essence, it seems there are only a few factors that you use in analyzing what is going on in the organization, and you are usually right.

  • Planning is a mysterious activity. After a few iterations, every relationship with reality seems to be based purely on coincidence.

  • Our organization seems to be in a constant state of reorganization.

  • Creativity seems to come from the smaller business units, far away from headquarters.

  • The more I dig, there is always one additional layer of complexity.
The more you recognize these situations, the more you confirm your organization is a complex adaptive system, as they translate into the properties that are associated with it: emergence, co-evolution, sub-optimal, requisite variety, connectivity, simple rules, iteration, self-organizing, edge of chaos, and nested systems.

The Organization as a Living Organism

Examples of complex adaptive systems include markets, organizations, ecosystems and most living organisms, such as human beings. To zoom in on one particular parallel: organizations can be compared with living organisms. Like people, organizations are born, grow up, and die. Some barely grow up; they die young and irresponsible. Other organizations mature and grow old and wise. Like people need oxygen to breathe, organizations need cash. Over time, organizations expand and sometimes contract, like people who gain weight and diet when necessary. Organizations, like people, create children in the form of new activities and business units that sometimes spin off into other activities and units.

There is an academic field focused on this approach, organizational behavior (OB). OB is the study of how people behave within organizations and, one level up, how organizations behave within a complete social system. OB relies on a system approach for the study and application of knowledge about how people, individuals, and groups act in organizations. This is the process of understanding how things influence one another within a whole.

In fact, groups (and as a consequence organizations) can display their own behavior, without being able to directly track this back to the behavior of a single person or element. Consider this famous experiment that has been described in many different ways over the years. Eight monkeys are put in a cage, with a bunch of bananas hanging at the top of a stairway. After the first monkey climbs the stairs, the other monkeys are sprayed with water. It doesn’t take long before any monkey climbing the stairs gets beaten up by the others. Then, one monkey gets exchanged for another, who gets beaten up the moment he attempts to climb the stairs, without knowing why. Another monkey gets exchanged, and the same thing happens. After all monkeys are exchanged, no monkey dares to climb the stairs to get to the bananas, without a specific reason for avoiding the stairs. The group displays behavior as a whole.

Consider another example. The US Navy has multiple aircraft carriers, and the largest of these are called super carriers. Between five and six thousand people work on such a ship, and every 18 months the majority of staff is replaced. Despite the heavy job rotation, every one of these super carriers is known for having a distinct culture, independent of the behavior of each of its individual crew members.

Organizations, being complex adaptive systems, have unique behaviors and develop a collective intelligence over the years, again independent of each of its individual members. The collective intelligence of an organization can be seen as the body of knowledge about the business and its market that is stored within the organization and somehow gets passed on to new organization members.

Taking it Waaaaay Further: Collective Consciousness

French sociologist Emile Durkheim (1858-1917) and Swiss psychiatrist and psychologist Carl Jung (1875-1961) took the idea of collective intelligence a few steps further and came up with the idea of collective consciousness. This is the collective sentiments, morals, conceptions and perceptions that the average citizens within a certain community share. People are bound to the norms and values of their environments, in which – as social beings – are doomed to exist. Personal behavior cannot be explained without the context of the environment from which they come. Jung came from a different angle. He discovered that people around the world share rituals, without being connected to each other. According to Jung, these rituals come from primal experiences that are passed on.

Inferring from here, it is no coincidence that religions have some version of what Christians call the Ten Commandments. Morals are roughly the same all over the world. You shall not kill, you shall not steal, and so forth. You can argue that morals are meant to protect the collective consciousness, as it cannot live without its inhabitants – human beings.

Here is where we need to be careful. When googling “collective consciousness,” you are never more than two mouse clicks away from an esoteric “new age” approach, or references to “The Secret” that explains how everything in the cosmos is connected. Interesting perhaps, but not usable for our purposes here.

Still, facing the danger to float away on a pink cloud, it is important to examine the idea of a collective consciousness a little. It is a logical and evolutionary thought. The first many centuries of mankind revolved around a collectivist approach. Survival, poverty, religion. The industrial age changed it all, and we live in individualistic times now, focused on wealth, self-actualization and science.

But individualism leads to suboptimal results. It has been successful so far because prosperity took away so many constraints that the suboptimal nature of everyone creating the maximum result for their own good alone did not limit growth. But in nature, individualism is not looking for synergistic results where the total result is more than the sum of the parts. The logical path of synthesis between the collective and the individual view is to find a way to connect individuals (despite the current countertrend of social and economic polarization).

Moreover, the concept of collective consciousness solves an important issue of philosophy: the kiss of death of postmodernism. In order to explain what I mean here, I need to take a few (big) steps back.

Man was the First Modeler

When mankind became conscious, it meant humans could distinguish themselves from the world around them. But there was a price to pay. It also means that we, human beings, are not fully part of the world anymore. All we can do is perceive what is going on around us and try to make sense of it. You could say that man was the first modeler.

Since the beginning of philosophy, people have tried to build a mental model of the world based on truth. We would identify objects, label them with names, and come up with all kinds of attributes to describe those objects. For instance, a cow has four legs, it eats grass and it has multiple stomachs. It can have a black and white pattern, but can also be brown. It has a certain DNA structure, and all these things together create an object called cow. This type of thinking lasted until into the Age of Enlightenment, and science is still full with it. The universe acts according to a certain set of rules, and all we have to do is discover them. Still, there is plenty to discover, even in hard sciences. For instance, to explain the increasing pace of expansion of the universe, the vast majority of required matter to fuel that expansion is simply unaccounted for. There are forces at work we don’t even know yet.

But in the twentieth century there was a turnaround to something we now call postmodernism. We realize that all we can know comes through our senses. As our senses have limited capabilities, our model of the world is limited by nature as well. We cannot state that a cow has certain attributes. We can only say that we see something that so far we have called “cow” and that we perceive some common elements. All attributes have moved from what we thought to be an object and a part of reality to a categorization in our mind. The things we see in what we call a cow (so far). Nothing more. Truth has disappeared in this way of thinking and has been replaced by a “sufficiently shared view.” The shifting of attributes to label and name things is not very uplifting. It means we are all alone in our perception of the world. Furthermore, I find it practically unacceptable: How can we function? Even a sufficiently shared view is not helpful because all we can do is perceive that. Postmodernism is a death trap. Once you have the thought that there is only perception, there seems to be no way out. And if you don’t agree with me, that is just your opinion.

The idea of collective consciousness offers a way out. It combines the ideas of truth and perception. Perhaps indeed all attributes of an object or concept exist only in the mind; but if they are shared through a collective consciousness, there is a process of replication and inheritance that makes sure we all have a similar way of perceiving and categorizing. As we are complex adaptive systems, we may have different ways of qualifying things. Some see a cow as a steak-to-be, others see it as something to trade or to get milk from. Some like cows and others don’t.

This process replication and inheritance doesn’t have to be “magic,” in which we are all “tapped into” a greater structure. British scientist Richard Dawkins came up with the idea of memes. A meme is an idea, behavior or style that spreads from person to person within a culture. Being an evolutionary biologist, Dawkins observes that this process is constantly evolving, like trends. Ideas, behaviors and styles change over time and keep spreading. However, the process is somewhat different than in the well-known children’s game where the first child in a circle whisper’s a sentence in the next child’s ear. At the end of the circle, the sentence has changed completely. Because everyone is communicating ideas all the time with everybody else, the story ”averages out,” and a good level of common truth emerges.

In this sense, the collective consciousness is an even better idea than both classical truth and postmodern perception. Truth evolves over time, like a complex adaptive system itself. The collective consciousness connects people, but there is also room for alternative and differing opinions.

But… time to return to earth. .

And Back to Business

I may have taken it a bit too far. Too airy-fairy perhaps. So… time to return to earth.

If organizations possess a body of knowledge that is transferred like memes and creates a collective consciousness to which members of the organizations belong and identify with, it would be great if we could somehow capture that with technology. Technology’s aim is to augment human capabilities (at least that is one view on the essence of technology), and this would be a huge augmentation of capabilities.

Knowledge management (KM), popular in the 1980s and 1990s wasn’t “it.” Although no one ever said that KM should be a top-down process, this is how it turned out in many cases. The project team and the team running KM were responsible for uploading all documents, feeding the search instruments, creating a central taxonomy and so forth. And, like most information management initiatives, KM highly mimicked the corporate hierarchy. Not ideal for creating a maximum buy-in throughout the organization. As most people already have their informal networks to get the information they want, there really is no need to use the corporate system. Corporate intelligence doesn’t equal collective intelligence.

The bottom-up approach was also often not very successful. Why would knowledgeable employees contribute to a knowledge management system and “give away” the asset that makes them unique and valuable? In fact, I heard of one example of a consulting firm that tried to promote its knowledge management system by publishing a top 10 of contributors and rewarding them for their contributions. With adverse effects. The list became known as the “losers’ list” and the “bench-sitter list,” consisting of people who obviously had time to do all this, in contrast to the important consultants who did the “real” work.

“Connecting individuals,” one of the central ideas of the collective consciousness, is also the core of social media. It would certainly explain the popularity of social media worldwide. Twitter and Facebook have become an integral part of our lives and give us a sense of security from being connected with the people around us. Social media shows what is important to others, and it leaves it up to our imagination what to do with it. In essence, social media platforms are superconductors for memes.

But are social media platforms the right tools for creating “wisdom of the crowds,” to use a more modern term for a collective intelligence? The data coming out of social media is unstructured, fragmented, usually highly ambiguous and lacking context. Social media may have most of the characteristics of a complex adaptive system; but social media, as they operate now, are also not “it.” Social media platforms are perhaps sufficient for private use for the time being, but they need more attention when using them in business. We need to find a way to fuse the structure of knowledge management with the dynamics of social media.

Simply adopting existing social media platforms and implementing them within your organization under the label of “Enterprise 2.0” is not enough. Any democratization of communications will lead to a quality-leveling effect, concurrent with massive expansion of communications. There most likely will be more knowledge sharing through forums where employees can discuss with each other, but that is still a few steps removed from a collective intelligence, let alone a collective consciousness.

First of all, there is still a need for focus and leadership. New strategies and ideas won’t come “automagically” through the wisdom of the crowds. A new style of leadership needs to be accepted. Influence and power in a networked environment doesn’t necessarily come from the ones at the top, but from the ones that are best networked and possess the best insights and credibility. Influence comes from different sources. But still it takes leadership to channel discussion to make sure all aspects are covered. Discussions through social media often lead to new and unexpected angles, but not necessarily to all relevant angles.

Leadership is also needed to keep focus in a discussion. A bottom-up and widely democratized discussion digresses easily. This is fine in blog comments, on Twitter, Facebook and other social media for private use, but not very helpful within an organization that wants to stimulate a collective intelligence and consciousness. A conscious entity is able to imagine a situation, a future, and plan a way towards that future. This is also the essence of strategy within an organization. An unmoderated discussion easily leads to what is called strategic drift. If strategy is seen purely as a process of emergence (which is one school of thought in strategy), taking an incremental approach typically causes the organization to easily drift away from its original direction. Or even worse, the organization gets nowhere because the discussion is all over the place and nothing gets implemented.

Bringing an action orientation to the discussion also requires leadership. Particularly in broad discussions, covering a wide variety of angles, there is a danger of analysis paralysis. This means no decisions are taken, as the phase of gathering and interpreting information is endless in nature. At one moment, the decision needs to be taken to stop discussing what to do, and either start discussing how to do it or hand out marching orders to get stuff done. (Even in today’s world, it eventually boils down to this.)

But understanding how to use technology is not enough. The technology itself needs to evolve as well to add to the creation of a collective consciousness. The word “consciousness” is direct from the Latin word “conscius,” which means “having joint knowledge with another.” In philosophy, if not debated, the term is used to describe experiencing perception and being aware. This means we should not only have the means of sharing information and knowledge with each other, as we do now using social media, but we should also get feedback on that process so we are aware of the process of knowledge sharing and its effects.

We can only imagine the analytical power that social media companies such as Facebook, Google and Twitter use to data mine and text mine the enormous amounts of data. On a very personal basis, they know what we talk about and with whom we talk about it. They know the brands we like and even capture significant amounts of our online behavior outside the use of the social media. On an aggregated level, this could help with overall sentiment analysis (how we feel about the economy, how accurately we predict the next winner of The Voice, or if we can predict social unrest in a particular neighborhood or country). This analysis could also provide insight into how common memes have become to see if they represent a common truth. The problem with these analyses is not that they are not available; the problem is that they are not aimed at the public (with the possible superficial exception of Twitter trending topics). They are aimed at the true customers of the social media companies, the advertisers. These analyses certainly add to a high level of intelligence, but their current value lies in their exclusivity, not in their collectiveness. It is not in the best interest of the current business model of the social media companies to change this and open up extensive analytics to the public, sharing the created consciousness.

In conclusion, business needs to get this right for its own sake, leveraging knowledge across the organization and creating a strong culture and body of knowledge less dependent on the specific people inside the organization. But moreover, business needs to get it right from a social responsibility point of view. Although the popularity of social media has been driven more by the consumer market than by business, the chance that business will drive the innovation in creating social media feedback is greater, simply because there is more alignment in the business model.

After having done that, the enormous wealth and power of social analytics should be used for the good of society, actively building the collective consciousness. I am sure there is a great business in doing that.

  • Frank BuytendijkFrank Buytendijk

    Frank's professional background in strategy, performance management and organizational behavior gives him a strong perspective across many domains in business and IT. He is an entertaining speaker at conferences all over the world, and was recently called an “intellectual provocateur” and described as “having an unusual warm tone of voice.” His work is frequently labeled as provocative, deep, truly original, and out of the box. More down to earth, his daughter once described it as “My daddy sits in airplanes, stands on stages, and tells jokes.” Frank is a former Gartner Research VP, and a seasoned IT executive. Frank is also a visiting fellow at Cranfield University School of Management, and author of various books, including Performance Leadership (McGraw-Hill, September 2008), and Dealing with Dilemmas (Wiley & Sons, August 2010). Frank's newest book, Socrates Reloaded, is now available and is highly recommended. Click here for more information on how to get your copy today.

    Editor's Note: More articles and a link to his popular blog are available in Frank's BeyeNETWORK Expert Channel. Be sure to visit today!

Recent articles by Frank Buytendijk



 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!