Personal InfoCloud

Lack of Understanding Begins Where?

“We can't solve problems by using the same kind of thinking we used when we created them.” ~Albert Einstein

Much of the lack of understanding with social software today is mistaking what seem like simple Web 2.0 tools and not understanding the depth of thinking and understanding from a technical, interaction design, and deeper understanding from a social science perspective of what is needed. Many Web 2.0 services rarely get into the 3rd stage of “Mature Social Tools”. When you bring this understanding into organizations and their needs for vastly improved communications, social interactions, collaboration, and efficiency needed the Web 2.0 model doesn't really get you far, nor help you prepare for what will come. (It is not that Web 2.0 offerings are not capable, it is that if they are even moderately successful they are dealing with many millions of users and keeping their offerings running with more simple social interactions and needs has them completely occupied).

Claiming your tools and services are like Web 2.0 tools and having them actually be rather equal to the lack of depth Web 2.0 products like Facebook have, becomes a pill filled with poison that once swallowed will release over time. The problem is less with to do with Web 2.0, but how things progress within fixed populations beyond the capabilities and needs (limited by volume and scale of resources needed to handle the volume of Web 2.0 services). Think of the fishing industry and the practices needed for fishing at massive industrial scale and optimizing skill of fly fishing and sustainability.

The Axis of This Model

Along the y-axis is the number of people participating in the service. As this increases the need for individuals to manage relationships and interactions increases. Along the x-axis are the number activities, which can be: Conversations, media shared, ratings, documents, short and long writings, annotations, organizing (curating) what exists in the system, etc.

Optimally the service will have growth that progresses in a relative balance between people participating and activities over time. If the balance has many people and few activities (or range or activities around subjects or tool types, see the differentiation between collective and collaboration, which doesn't include community/group distinct needs) the system will be really narrow and seem like their is little activity or action and perceived value dissipates and the usual result is decreasing visits and use. If the services has a relatively low number of people participating and a lot of activity the outcome is usually a very narrow view and lack of breadth of understandings, which limits the perception of what subject matter or activities types happen there.

What Are The Scaling Stages?

Personal

This stage is firmly set in the simple (a passing or deeper knowledge of Dave Snowden's Cynefin framework will understand the framing help). All social tools and services start their useful existence with personal value. They are offering where people place what they know or see where they can come back to it easily, as well as share with others, who will / may eventually find it. This clarity of understanding the personal impact was really clear when Delicious started. Joshua Porter actually called this the “Delicious Lesson”. The personal also helps initially frame what you have interest in and captures it, provides seeing others to connect with to initial share with and follow, provides a means to hold onto connecting with people, and hopefully allow people to see this in their own contextual lens. There is very little social interaction as things start out. It takes work of planning, engaging, and managing the initial social interactions. Community managers (instigator and evangelists) are essential for helping people into this first stage and get the whole moving toward the next stage. Problematically many services under provide for the needs and capabilities of the personal needs, not only for enabling initial uses, but for more valuable needs as the services mature. Seeing and managing who a person connects with and why along with actions taken in the system (accounting for time, cycles, and patterns) is a real need which helps people not only use the services but see the value they get from it.

Serendipity

This second stage still has most of its focus on the simple, but toward the edges of the next stage that shifts. Once the service gets more people using it and the activities increase things move from a heavily personal focus to one that is more social. The social interactions are more serendipitous than planned interactions as people aggregate and interact mostly through stumbling onto are being guided to subject mater areas of interests, groups or areas where conversations and objects related to the subject are shared and conversations around them happen (social objects).

In this stage the interactions between people are often echo their connections to people and interests that exist prior to using the services. The information flows are still rather manageable, but start edging into flows with some serious volume and velocity at times, which creates and information density to me dealt with. As the activities increase, particularly across groups and subject matter affinities and needs the need for tools to help with various roles people have (either roles that emerged, take on out or need or adeptness, or are have been assigned) is needed. The roles, other than admin and guide, are still mostly light. The managing of information and connecting it to where it is needed is what surfaces here as activities grow.

As time increases and the people participating and activities increase (as expected) things shift to being simple to more complicated given the number and variance of people interacting with each other. Managing connection and what is shared with whom starts to be seen, as does the reality that open social platforms can greatly hinder social interactions (no matter the culture) as the realization that there is something to Robin Dunbar%s [magical] number. As this happens the impact of the organizations overarching culture starts to have an impact and the selection of the tools and services for the social interactions comes in to clarity, whether the right choices were made and implemented to easily integrate with it or clash.

Mature Social Tool

The mature social tool stage the complicated realities of human social interactions comes into play, as well as the need for managing and filtering information flows. Most often organizations hit this stage in 6 months to 2 years. The lovely “if information is important it will find you” theory falls from a working practice to myth here as does they never valid 1/9/90 rule. Information and connections with people get lost and fuzzy. Keeping what is needed and valuable near is essential. It is here most often that people managing the services and tools in organizations state, “What they hell did we do? Do we have the right tools and services?” Many times the answer is no they don't have what they need as they didn't see this part of the picture and reality coming. Also they didn't plan budget and resources for this (it was supposed to just work, right).

It is also in this stage that it is really clear different parts of the service have matured at vastly different rates. Some of it is individual people maturing at faster rates. The accelerated maturity is not only with individuals but groups, subjects, use patterns, roles, etc. This inconsistency of growth is normal, yet it continually seems to surprise people. The reality is there are various types of people, whom these tools hit a need and map tightly to their activities and perceived way forward. Rarely does accelerated maturity of use have much to do with age (the myth that it is young people who take to these tools really becomes clear here as well). Matching lack of resources and pain because that with other solutions is a much stronger driver when the services ease those pains.

The mature social stage is also where the “best practices” considered and possibly used earlier surface as possibly not the best way forward and may have lead to things more problematic than not optimal outcomes. Each organization not only has its own culture, but sub-cultures, but its own ways of doing business on top of the social environment and cultural behaviors. Understanding what the levers and myriad of potential options the possible outcomes that come from their use is an incredibly valuable approach. Combining approaches and methods from these many options will enhance the complications, which needs the ability to have people who can understand and see the components and break down what influences can be attributed to where. It is very much an iterate, test, monitor, and iterate practice all while realizing what doesn't work in one scenario may be brilliant in others.

The value from much of the social web understandings derived from what people thought they saw in Web 2.0 offerings runs out and the practice of copying features and functionality from that realm has run its course due to limitations mentioned above. The practices and services are similar, but the massive scale that Web 2.0 services handle has them focussing on volume and quantity of interactions, not the honed qualitative needs in organizations. Facebook doesn't care that people are sharing important knowledge for other to benefit from as long as people are interacting and using their service. Sharing and honing those understandings and being able to refind them as needed in an organization is an essential and has deep value over time.

The mature social tool stage is where search is needed to find things and social search (in theory) should work well (that often isn't the case as search for the most part hasn't caught up yet). There is enough content and enough people interacting to see a rich ecosystem ready to see the benefit of these service become really valuable. This can happen, but it becomes difficult. There are no best practices that work here, there are guides and series of “it depends” scenarios and lenses to work through to good (if not hopefully better) outcomes. The number or roles and tools matching those role's needs are needed for many using the service, but at the same time keeping the interfaces easy to use as they were in the earlier stages (think of most role playing games that start with simple interfaces that are easy to use to accomplish what is needed, but over time and proven adeptness at using them more complicated tools and interfaces slowly evolve that match the mastery, roles, and skills needed (Lithium community platform (for outside the firewall) does this amazingly well, but doing this is something that takes incredibly deep skill and understanding).

It is also in this stage that information overload really can kick in. Connecting the information and knowledge to people and areas in the system that need it can become a challenge. What seemed to be a reality of a single culture in the organization is seen as more complicated with the multitude of sub-cultures with their own understandings, contexts, terms/vocabularies, and expectations. Not only do non-emergent taxonomies have problems here, but search does if it doesn't account for the social implications and influences underlying the content and needs.

By this point the realization that an open social platform didn't work there are now many smaller groups that are fully or partly closed off. The key is to embrace this understanding and work to build synonym repositories and bridges of understanding between the sub-cultures and divergent practice areas. The collective whole that is emerging becomes difficult to work with, but it can be done. The scale and needs that emerge out of this can begin to look like enterprise resource management services, but the components are not as stable and as predictable, they are human and social.

Focussing on the complicated components in all of this is a task. It can be done and taking the multitude of complicated steps, conditions, and interactions (software and social, as well as social software interactions) into account and breaking them down into smaller more manageable components through depth of understanding and experience can be done. Having not only a good understanding of broader social network interactions helps greatly, but understandings at the social interaction design level for the much smaller scale interaction needs is essential as well. The interfaces and needs of the service will be drastically different than what is needed earlier in the stages.

Even with some mastery of this stage the growth of people and actions over time will shift from being complicated to complex. Hopefully, the complicated needs are being identified and needs relating to the complicated needs are helping to address the issues at hand. Longitudinal understandings of use and patterns is needed to help iterate and meet needs.

Complex Social System

The complex social system is where things move toward emulating actual social systems in the world around us. Understandings that are central to urban planning and understanding healthy societies at scale, as well as using well worn research and theories for how the complex organisms known as societies interact. (Dave Gray has picked up on this and included it in the Connected Company post, which is worth your time to read.) There are few universal understandings of what people do that will consistently apply. The use and emergent uses of the services that happen in this stage will be quite different and the tools and patterns for managing things that worked in earlier stages will not work as well. External influences (influences outside of the cultures or are emergent and not planned) will impact use and value. Often it is these emergent uses that have the highest value, but they can also be problematic. It is also essential to understand how modifying the whole of the system and service to embrace these emergent patterns will impact.

There are no best practices and never will be. It takes identifying and understanding the individual influences (there are often many) and their place in what is occurring in small samples (rarely do large emergent patterns behave or happen consistently across the organization (although it can)) to get better clarity.

Knowing this stage is coming and being aware of the patterns indicate this emergent and divergent stage is really helpful as early as the initial planning stages. Indications where and how these patterns are emerging can be seen very early and they can be confused for mainstream use, which changes the whole of the system and skews it against easy considered use in the earlier stages. This isn't something to understand and worry about later, it needs to be something that is firmly in mind with people who not only grasp it, but can ascertain its existence and work through the myriad of considerations that will be needed to work through to best prepare and adapt for it.

Tools and services are not exactly here just yet. There are some that could be close, but it all is dependent on need, problems, and the underlying complications that lead to the complexity. There are also many examples for services identifying emergent patterns and behaviors and adapting for them or just letting them be. Things like hashtags in Twitter are an example of embracing the emergent patterns, but it was and is an edge user pattern. This past week Socialcast took the steps to further adapt their system to take hashtag and enable design patterns that helped it be far more usable and understandable to mainstream core users (I think I may know some people who worked on that and bravo all around).

Redundancy

Never the less, this brings-up an important point: Redundancy. One lesson I learned many years ago related to the web (heck, related to any thing digital) is it will fail at some point. Cloud based services are not immune and the network connection to those services is often even more problematic. But, one of the tenants of the Personal InfoCloud is it is where you keep your information across trusted services and devices so you have continual and easy access to that information. Part of ensuring that continual access is ensuring redundancy and backing up. Optimally the redundancy or back-up is a usable service that permits ease of continuing use if one resource is not reachable (those sunny days where there's not a cloud to be seen). Performing regular back-ups of your blog posts and other places you post information is valuable. Another option is a central aggregation point (these are long dreamt of and yet to be really implemented well, this is a long brewing interest with many potential resources and conversations).

With regard to Delicious I’ve used redundant services and manually or automatically fed them. I was doing this with Ma.gnol.ia as it was (in part) my redundant social bookmarking service, but I also really liked a lot of its features and functionality (there were great social interaction design elements that were deployed there that were quite brilliant and made the service a real gem). I also used Diigo for a short while, but too many things there drove me crazy and continually broke. A few months back I started using Pinboard, as the private reincarnation of Ma.gnol.ia shut down. I have also used ZooTool, which has more of a visual design community (the community that self-aggregates to a service is an important characteristic to take into account after the viability of the service).

Pinboard has been a real gem as it uses the commonly implemented Delicious API (version 1) as its core API, which means most tools and services built on top of Delicious can be relatively easily ported over with just a change to the URL for source. This was similar for Ma.gnol.ia and other services. But, Pinboard also will continually pull in Delicious postings, so works very well for redundancy sake.

There are some things I quite like about Pinboard (some things I don’t and will get to them) such as the easy integration from Instapaper (anything you star in Instapaper gets sucked into your Pinboard). Pinboard has a rather good mobile web interface (something I loved about Ma.gnol.ia too). Pinboard was started by co-founders of Delicious and so has solid depth of understanding. Pinboard is also a pay service (based on an incremental one time fee and full archive of pages bookmarked (saves a copy of pages), which is great for its longevity as it has some sort of business model (I don’t have faith in the “underpants - something - profit” model) and it works brilliantly for keeping out spammer (another pain point for me with Diigo).

My biggest nit with Pinboard is the space delimited tag terms, which means multi-word tag terms (San Francisco, recent discovery, etc.) are not possible (use of non-alphabetic word delimiters (like underscores, hyphens, and dots) are a really problematic for clarity, easy aggregation with out scripting to disambiguate and assemble relevant related terms, and lack of mainstream user understanding). The lack of easily seeing who is following my shared items, so to find others to potentially follow is something from Delicious I miss.

For now I am still feeding Delicious as my primary source, which is naturally pulled into Pinboard with no extra effort (as it should be with many things), but I'm already looking for a redundancy for Pinboard given the questionable state of Delicious.

The Value of Delicious

Another thing that surfaced with the Delicious end of life (non-official) announcement from Yahoo was the incredible value it has across the web. Not only do people use it and deeply rely on it for storing, contextualizing links/bookmarks with tags and annotations, refinding their own aggregation, and sharing this out easily for others, but use Delicious in a wide variety of different ways. People use Delicious to surface relevant information of interest related to their affinities or work needs, as it is easy to get a feed for not only a person, a tag, but also a person and tag pairing. The immediate responses that sounded serious alarm with news of Delicious demise were those that had built valuable services on top of Delicious. There were many stories about well known publications and services not only programmatically aggregating potentially relevant and tangential information for research in ad hoc and relatively real time, but also sharing out of links for others. Some use Delicious to easily build “related information” resources for their web publications and offerings. One example is emoted by Marshall Kirkpatrick of ReadWriteWeb wonderfully describing their reliance on Delicious

It was clear very quickly that Yahoo is sitting on a real backbone of many things on the web, not the toy product some in Yahoo management seemed to think it was. The value of Delicious to Yahoo seemingly diminished greatly after they themselves were no longer in the search marketplace. Silently confirmed hunches that Delicious was used as fodder to greatly influence search algorithms for highly potential synonyms and related web content that is stored by explicit interest (a much higher value than inferred interest) made Delicious a quite valued property while it ran its own search property.

For ease of finding me (should you wish) on Pinboard I am http://pinboard.in/u:vanderwal

----

Good relevant posts from others:

• Why You Should have a Website - Jeremy Keith
• We can save Delicious, but probably not in the way you think
• Delicious in Purgatory :: TechCrunch

Personal

The first step or home base, is more of a state for beginning, is the the personal space and repository. Sadly, this is the ugly step child that is very often missed in many tools/service offerings. The place were a person has a view of their resources, which is mapped in their context and needed representations to make sense with the least effort. This is the view with things they need to see surface (from their perspective and from others) and from where they jump to interacting with information, objects, tasks, and others.

Sparks (Ideas Shared)

The first step often comes from asking questions simply and easily and quick easy responses, or sharing quick notes and ideas that get feedback and interest. Many times this is done efficiently in micro sharing services like similar to Twitter but with a grasp of needs organizations have (Socialtext Signals or Socialcast are solid options to consider). But, other options, including blogs and discussion forums have the capability of doing this as well.

With sparks of ideas they need to have the ability to be found so to be responded to, aggregated, or even shared to ensure the right people see them and can interact. There is a wide breadth of types of things that flow through micro sharing services, but many will resonate, inform, or inspire others. But, quite often they get solid conversations flowing across a broad cross section of people and locations.

Campfire (Gathering of Others with Interest)

From the spark of inspiration many others with interest or affinity gather to discuss and the spark turns into a campfire. Stories are told and fuel is added to the fire. Honing of the ideas and gather inspiration, information, and content from broad sources and view is then curated and honed to some degree.

The tools needed for the campfire stage must allow from much broader conversation than the limited spark stage. Limiting the room around the campfire to those with strong interest and affinity helps keep the focus, but also these people will likely have the deepest reserves of fodder for the conversation and a wide variety of perspectives and resources they can tap ready at hand. Longer conversation and curating all that is gathers are the prime focus. Curation through tagging is often incredibly helpful (being able to tag so to aggregate and curate ideas from the sparks stage is highly important).

Bonfire (Broader Interest Gathering)

Once the ideas have been fleshed out and framed to some degree and curated to control scope the discussion turns into a bonfire. Bonfires, while much larger still need to be controlled and maintained or they get out of control and things get dangerous. At this stage broad viewing for healthy feedback and discussion, including highlighting things that have been missed, what works well, what doesn't work well, etc. are the key focus. This is the time to get understanding and direction that hones and shapes everything that is possible. It is also used to add to what has been gathered and curated in the campfire stage so to iterate on it.

Torch (Honing for Broad Use & Replication)

Lastly, is the torch stage. This is easy to handle, easy to replicate, and is safe. This requires Real Collaboration to work through the conflicting ideas and negotiate as well as intelligently work toward one final output. These final outputs can be white papers, new processes, new guidelines, new products, etc. But, the point is there is one (just like artists collaborating on a statue there is only one statue, not many and all through differences have been worked through to one salient solution).

We Can't Find Anything

There is nothing more common that I hear from an organization about their intranet and internal information services than, "We can't find anything." (Some days I swear this is the mantra that must be intoned for an organization to become real.)

There are many reasons and potential solutions for improving the situation. Some of these involve improved search technologies, some improved search interfaces, or But, understanding the cost of this inefficiency is where I find it is valuable to start.

The first step after understanding you have this problem is to measure it, but most organizations don't want to pay for that they are just looking for solutions (we all know how this turns out). The best method I find is walking through the broad understandings of the cost of inefficiencies.

The Numbers...

At Interop 2009 I presented "Next Generation Search: Social Bookmarking and Tagging". This presentation started off with a look at the rough numbers behind the cost of search in the enterprise (see the first 16 slides). [I presented a similar presentation at the SharePoint Saturday DC event this past week, but evaluated SharePoint 2010's new social tagging as the analysis focus.]

Next Generation Search: Social Bookmarking and Tagging

View more presentations from Thomas Vander Wal.

Most of the numbers come from Google white papers on search, which gets some of their numbers from an IDC white paper. I also have a white paper that was never published and is not public that has slightly more optimistic numbers, based on the percentage of time knowledge workers search (16% rather than the Google stated ~25% of a knowledge workers time is spent searching). There are a few Google white papers, but the Return on Information: adding to your ROI with Google Enterprise Search from 2009 is good (I do not endorse the Google Search Appliance, but am just using the numbers used to state the problem).

I focus on being optimistic and have I yet to run into an organization that claims to live up to the optimistic numbers or total cost of inefficiency.

• Few organization claim they have 80 percent of or better success with employees finding what they need through search
• That is 80 percent success rate
• Or, 1 in 5 searches do not find what is they were seeking
• A sample organization with 500 searches per day has 100 failures
• An average knowledge worker spends 16% of their time searching
• 16% of a 40 hour work week is 1.25 hours spent searching
• 20% (spent with unsuccessful searches) of 1.25 hours a week is 15 minutes of inefficient productivity
• At an average salary of $60,000 per year that leads to $375 per person of inefficient productivity
• Now take that $375 per knowledge worker and multiply it by how many knowledge workers you have in an organization and the costs mount quickly
• An organization with 4,500 knowledge workers is looking at a inefficiency cost of $1,687,500 per year.
• Now keep in mind your knowledge workers are you most efficient at search
• Many organizations as a whole are running at 40% to 70% success rate for search

We Know We Have a Costly Problem

This usually is enough to illustrate there is a problem and gap with spending time resolving. The first step is to set a baseline inside your organization. Examine search patterns, look at existing taxonomies (you have them and use them to some degree, yes?) and work to identify gaps, look at solutions like tagging (folksonomy) to validate the taxonomy and identify gaps (which also gives you the terms that will likely close that gap). But get a good understanding of what you have before you take steps. Also understand the easy solutions are never easy without solid understanding.

Evaluating what, if any taxonomy you have is essential. Understand who is driving the taxonomy development and up keep. Look at how to get what people in the organization are seeking in the words (terms) they use intend to find things (this is often far broader than any taxonomy provides).

Real Value

Having a fascination with economics and the “pure flow of information...” mantra I highly value information and the tools and services that provide the value chain of data, information, and knowledge. These digital tools were not the easiest things to work with for many people and it has always been a passion to have the tools and services work better. More optimally, so people could have better access to information so to help them make smarter decisions around things that matter(should we find a new supplier, do we have a problem, do I need a coat, does our packaging need to be weather resistant, etc.).

What matters and what is work and what is personal is a very blurry line, but having the information and ease to access it so we are smarter in making decisions it the key. It comes down to efficiency, which is highly related to ease of use.

Real Populations

What fascinates me most with inside the firewall and always has is the need to understand how people use (can’t use) the tools that have been built or deployed for their use. Things that are seemingly logical and intuitive from the developer and designer’s viewpoints are not on target with those in the organizations. When I started working managing, maintaining, building, and improving the tools and services people use it was inside the firewall as the web did not exist yet and the internet was still in its nascent stages, even if it had been around for 20 years already.

The groups of people I working with needed to use these tools and services to perform their job as the paper and non-technical means of performing their tasks were replaced by computers or were never possible with out the power of digital computations. What was true then with dealing with the populations of co-workers and others inside an organization using the the tools and services is still true now, success of a product is measured by its percentage of use from those who must use it, efficiencies gained, lack of bugs, and improved time to complete tasks.

Web projects seemed to lose these values as it was easy (relatively) to get a few thousand, hundred thousand, or few million (over time) using a product or service. But, those services were only a small slice of the population, even a small slice of the population who needed a service like the one being offered.

Real Social

In the last five to eight years or so that truth around small slices of the populations using tools and services is never more relevant than around the flood of interest in social web sites and tools. Having built, managed, and iterated on intranet groupware and community tools for tens of thousands of distributed employees and business partners, I had great interest in seeing what happened with social sites on the web.

It was no surprise to me when variants of the web’s social tools and services started coming inside the firewall that adoption was less than optimal, because these social tools were being honed and iterated on early adopters and assumptions that are very counter to the majority of the population (some 90% are outside of this early adopter trend using the tools).

Early on I learned the easiest means of getting adoption with tools and services is to emulate who things are done by people without technology mediating the tasks or flows. Regarding social interactions these is never more true.

Most of the social tools are not very social in the way that the majority of people are social. This is very problematic inside an organization because businesses and organizations are social by nature and must be to have any success. People must be social and interact with each other inside the organization (meetings, reviews, research, sharing findings, etc.) as well as to the outside with their customers and clients.

What many of these social tools, and business tools in general, have done is add friction to social interactions that are required by businesses to survive. These newer class of tools are moving towards emulating true human social interactions more closely, but we still have a long long way to go. Where the social web tools have fallen down is focussing on the early adopters, but in reality that is core group of people who come to these sites and services (services like AOL, Yahoo, and Facebook have over the years broken into more mainstream customer bases, but the customers are most often not using the really new “cool” stuff).  The lessons learned from most web social services often don’t work well inside organizations as they are not lessons learned from a full broad population, like the ones inside an organization.

Real Needs

Businesses and organizations have real needs for these social tools, as their organizations are quite inefficient and they know it. They know the value that these tools can bring and many have experimented with these tools in the past year or few, but have been stumped by lack of use and adoption.

Organizations are forever trying to optimally capture what they know (hence knowledge management interest), get information out easily to those who need it (portals), connect employees to each other (groupware), connect to customers and business partners more easily (B2B tools), and better connect the company to its employees (HR tools). All of these have received incredible funding and effort over the years. Some have decent payoffs to the organization (return on investment (ROI)), but rarely are they the large successes that had been promised or hoped for.  One of the big reasons is the tools got in the way.

Real Solutions

Getting the tools out of the way and allowing for people to interact as needed and as is comfortable is where success lies for tools and services in organizations. This is why I am passionate about this area and why I like focusing inside as not only do I see real solutions lurking in what has been done in what is called Web 2.0, but business and organizations see that same.

What is needed is using the understanding of organizations, the new tools, and marrying that to how real people are social and interact so to get to real optimal solutions.

Latency from Heavy Computational Requirements

What? I am hearing screaming from the engineers about the computational power needed to do this as well as the latency in this system. Design Engaged 2005 I brought up a similar scenario, within context of my Personal InfoCloud and Local InfoCloud frameworks called Clouds, Space & Black Boxes (a 500kb PDF). The key then as it is still is using location and people to build potential context and preprocess likely queries.

When my phone is sharing my location with the social contextual memory parser service that see I am quite near a book store (queue the parsing for shared books, favorited conversations with books, recent wish list additions (as well as older additions), etc. But, it is also at the time I usually eat or pick up food for a meal, so restaurant and food conversations parsed, food blogs favorited (delicious, rated on the blogs, copied into Evernote, or stored in Together or DevonThink on my desktop, etc.) to bring new options or remind of forgotten favorites.

Now, if we pull this contextual relevance into play with augmented reality applications we get something that starts bringing Amazon type recommendations and suggestions to play into our life as well as surfacing information “we knew” at some point to our finger tips when we want it and need it.

Inside the Firewall

I have been helping many companies think through this inside the firewall to have, “have what we collectively know brought before us to help us work smarter and more efficiently”, as one client said recently. The biggest problem is poor metadata and lack of even semi-structured data from RDFa or microformats. One of the most important metadata pieces is identity, who said what, who shared it, who annotated it, who commented on it, who pointed to it, and what is that person’s relationship to me. Most organizations have not thought to ensure that tiny slice of information is available or captured in their tools or service. Once this tiny bit of information is captured and contextualized the results are dramatic. Services like Connectbeam did this years ago with tags in their social bookmarking tool, but kept it when they extended the ability to add tagging in any service and add context.

Past Attendees have Said...

“A few hours and a few hundred dollar saved us tens of thousands, if not well into six figures dollars of value through improving our understanding” (Global insurance company intranet director)

From an in-house workshop…
“We are only an hour in, can we stop? We need to get many more people here to hear this as we have been on the wrong path as an organization” (National consumer service provider)

“Can you let us know when you give this again as we need our [big consulting firm] here, they need to hear that this is the path and focus we need” (Fortune 100 company senior manager for collaboration platforms)

“In the last 15 minutes what you walked us through helped us understand a problem we have had for 2 years and a provided manner to think about it in a way we can finally move forward and solve it” (CEO social tool product company)

Is the Workshop Only for Designers?

No, the workshop is aimed at a broad audience. The focus of the workshop gets beyond the tools’ features and functionality to provide understanding of the other elements that make a giant difference in adoption, use, and value derived by people using and the system owners.

The workshop is for user experience designers (information architects, interaction designers, social interaction designers, etc.), developers, product managers, buyers, implementers, and those with social tools running already running.

Not Only for Enterprise

This workshop with address problems for designing social tools for much better adoption in the enterprise (in-house use in business, government, & non-profit), but web facing social tools.

The Workshop will Address…

Designing for social comfort requires understanding how people interact in a non-mediated environment and what realities that we know from that understanding must we include in our design and development for use and adoption of our digital social tools if we want optimal adoption and use.

• Tools do not need to be constrained by accepting the 1-9-90 myth.
• Understanding the social build order and how to use that to identify gaps that need design solutions
• Social comfort as a key component
• Matrix of Perception to better understanding who the use types are and how deeply the use the tool so to build to their needs and delivering much greater value for them, which leads to improved use and adoption
• Using the for elements for enterprise social tool success (as well as web facing) to better understand where and how to focus understanding gaps and needs for improvement.
• Ways user experience design can be implemented to increase adoption, use, and value
• How social design needs are different from Web 2.0 and what Web 2.0 could improve with this understanding

More info...

For more information and registration to to Viget Lab's Social Design for the Enterprise page.

I look forward to seeing you there.

Overview

The last couple of years I have had many conversations with a broad selection of mid-sized to large organizations. Some of these are customers of mine or potential customers while others are conversations I have had, but all having the similar discussion about social tools in the enterprise. What follows is a collection of snippets from those conversations regarding Microsoft SharePoint 2007, most are not publicly attributed as they were not intended to be “on the record”.

One common element from all of the discussions is the frustration nearly all of these organization have with their experience with Microsoft SharePoint 2007. The comments are based on those spending one month to a year with the tool (the six month to one year club with tools offer best insight).

SharePoint does some things rather well, but it is not a great tool (or even passable tool) for broad social interaction inside enterprise related to the focus of Enterprise 2.0. SharePoint works well for organization prescribed groups that live in hierarchies and are focussed on strict processes and defined sign-offs. Most organization have a need for a tool that does what SharePoint does well.

This older, prescribed category of enterprise tool needs is where we have been in the past, but this is not where organizations are moving to and trying to get to with Enterprise 2.0 mindsets and tools. The new approach is toward embracing the shift toward horizontal organizations, open sharing, self-organizing groups around subjects that matter to individuals as well as the organization. These new approaches are filling gaps that have long existed and need resolution.

Broad Footprint

What SharePoint 2007 Does Well

Microsoft SharePoint 2007 seems to be in every enterprise I talk to, at least somewhere. It is used if a variety of different ways. When SharePoint is included with addition of Microsoft Office Online (MOSS) is a helpful addition for simple use of these older prescribed methods. MOSS is also good at finalizing documents that are the result of a collective, to group, to collaboration knowledge work process. MOSS and SharePoint are not great at anything but the last step of formalizing the document for distribution in another workflow.

A recent report from AIIM that was written-up by CMS Wire in “Study Finds SharePoint Primarily Used for File Sharing” states “47% use it primarily for file sharing (and/or as an internal Portal 47%)”.

How Did We Get Here?

There is one common point I have heard with nearly every company I have talked with over the last couple years, MS SharePoint 2007 is nearly ubiquitous in deployment. Nearly every organization has deployed SharePoint in some form or another. Many organization have tested it or have only deployed pieces of it. The AIIM survey reported by CMS Wire states: “83% currently use, or planning to use, SharePoint”.

Organizations either sent their IT out for training on SharePoint 2007 and/or brought in consultants to help build an implementation that fit their requirements. Most of the requirements IT departments started with were rather thinly informed, as they have nearly all stated after using SharePoint for a month, most realize after six months or so, their requirements are vastly different than what their initial requirements were, as they have learned more deeply about social tools in the enterprise.

Many who deployed SharePoint, thought it was going to be the bridge that delivered Enterprise 2.0 and a solid platform for social tools in the enterprise is summed up statement, “We went from 5 silos in our organization to hundreds in a month after deploying SharePoint”. They continue, “There is great information being shared and flowing into the system, but we don’t know it exists, nor can we easily share it, nor do much of anything with that information.” I heard this from an organization about 2 years ago in a private meeting and have been hearing near similar statements since. This is completely counter to the Enterprise 2.0 hopes and wishes they had for SharePoint. They were of the mindset that open sharing & having the organization and individuals benefit from a social platform.

MS Marketing’s Promise

The Microsoft marketing people seem to have performed their usual, extend what the product can do to the edges of its capabilities (and occasionally beyond) to map to customer stated desires. In 2006 and 2007 the advent of social computing on the web (Web 2.0) had entered the hormone raging stage gathering attention in boardrooms and IT departments who had been playing around with the ideas of bringing these tools inside the firewall in an official manner. The desire for social software to be part of the enterprise was an interest and desire.

The Microsoft marketing materials they focus on “collaboration and social computing”, which is more of a document management and workflow process tool that they put the more fashionable moniker on. But, it is this Microsoft marketing that engendered many organizations to the idea of the value and promise of social computing inside the firewall and Enterprise 2.0. Microsoft’s marketing legitimized the marketplace, but in typical Microsoft form did not exactly deliver on the promise of marketing.

Part of the promise of SharePoint is a malleable platform, which many developers who work across platforms complain is one of the least malleable and easy to develop on platforms. There are many constraints built into SharePoint and developers for SharePoint are not cheap. Development cycles for SharePoint as said to be about one third to half longer than most other options. At the Enterprise 2.0 Conference this past Summer in Boston, Lockheed Martin had a session demonstrating what they had built on top of SharePoint and it was quite impressive. But when asked about costs and resources, they said: “It took about one year, 40 FTE, and 1 to 5 million U. S. dollars. Very few organizations have those type of resources with availability to take on that task.

What Microsoft marketing did well was sell the value that social tools bring into the enterprise. They put the ideas in the minds of those building requirements (at a minimum to be included in pilot programs) as well as the values derived from using this new generation of social software inside an organization.

Multiple Micro-silos

At various conferences, across many industries, I have spoken at I have been asked to sit in on the SharePoint sessions, which turn into something like group therapy sessions (akin to group therapy in the first Bob Newhart show). There is much frustration and anger being shared as people try to resolve how to share information between groups and easily merge and openly share information once it has been vetted. These groups consistently talk about going directly to their Microsoft support & SharePoint Experts with these problems only to be told it is doable, but far from easy and may break some other things. Finding relevant information or even the inkling that something is happening in some group is nearly impossible. The promise of setting up ad hoc open groups by employees across silos is nearly impossible with out getting authorization.

Information Locked

One of the largest complaints is the information is locked in SharePoint micro-silos and it is nearly impossible to easily reuse that information and share it. Not only is the information difficult to get at by people desiring to collaborate outside the group or across groups, but it is not easily unlocked so that it can benefit from found in search. The Microsoft SharePoint model is one that starts with things locked down (focussed on hierarchies) then opens up, but unlocking is nowhere near as easy a task as it should be.

SharePoint Roadmap Marginalized Over Time

Where do people turn that have gone down the SharePoint route? Well most start by adding solid functionality they had thought SharePoint was going to provide or wished it had. SharePoint has acknowledged some of this weaknesses and has embraced outside vendors that make far superior products to plugin as components.

Some common social tool plug-ins to SharePoint are Socialtext, Atlassian Confluence, and Connectbeam (among with many others). Then there are those who build on top of Sharepoint, like Telligent and News Gator Social Sites. While others are more prone to full platforms that deliver much of the functionality out of the box, like Jive Clearspace.

Plug-ins Extending Functionality to SharePoint

Microsoft makes great promises, or hints at them in its marketing materials for SharePoint along the lines of social software in the enterprise. The first step many organizations take with SharePoint after realizing it does not easily, or even with an abundance of effort, do the expected social software components is to start getting solid proven services and start plugging them in. Many tool makers have taken their great products an made it quite easy to plug them into the SharePoint platform. Want a great wiki tool, not the horrible wiki “template”, then Confluence or Socialtext is added. Need a great social tagging/bookmarking tool that ties into search (this starts enabling finding the good information in SharePoint’s micro-silos), then Connectbeam is added.

This list goes on with what can be plugged-in to Sharepoint to extend it into being something it hints strongly it is quite capable of doing. What one ends up with is a quite capable solution, but built on top of one of the more pricy enterprise platforms. In most cases the cost of all the plug-ins together is less than the cost of SharePoint. It is from this point that many organizations realize all of these add-ins work wonderfully with out SharePoint (however, getting all of them to work together as easy plug-ins to each other is not always easy).

Full-Suites On Top of SharePoint

Another option that organizations take is to move in the direction of putting a fully functional social platform on top of SharePoint. Tools like Telligent and NewsGator Social Sites. These are options for those who find value in what SharePoint offers and does well (but and therefore getting rid of it is not an option), but want ease of development and a lower cost of development than is the norm for SharePoint. These full-suites also provide the ease of not having to deal with working through plugging together various different best of bread solutions (this really reminds me of the path content management systems went down, which was less than optimal).

Not only is the Lockheed Martin example of building on top of SharePoint an example of expense of that platform, but the recent AIIM survey surfaces high cost of development as a rather common understanding:

“Another area of interest is the required effort to customize SharePoint and integration other third-party solutions. In this case, 50% of survey respondents indicated custom solutions required more effort than expected (33% “somewhat more” and 17% “much more”). The integration challenges focused on a lack of training/documentation and integration with non-Microsoft based repositories and existing applications.” From CMS Wire: Study Finds SharePoint Primarily Used for File Sharing.

Fully Replacing SharePoint

There is a third option I have been running into the last year or less, which is removing SharePoint from the organization completely. I know of two extremely large organizations that are removing SharePoint from their organization this year (once these organizations are public with this I can be). The reasoning is cost and under performing as a social platform and what is does well is easily replaced with other solutions as well. In one instance I know the people who brought in SharePoint are being let go as well as the whole team of developers supporting it. I am hearing business operations looking into having their IT department find something that is meets their needs and were promised by IT that SharePoint was that solution. This was echoed by Lee Bryant via Twitter [http://twitter.com/leebryant/status/1099413469]: “[…]problem is many IT depts just don’t care - it is a simple ‘solution’ for them, not their users”

When removing SharePoint some organizations are going the piece by piece approach and stitching together best of breed or are going the route of full-service social platform, like Jive Clearspace. The cost per users of such solutions is less, the time to install to up-and-running fully is reportedly a about a third and maintenance staffing is also reportedly lower.

SharePoint is not Enterprise 2.0

What is clear out of all of this is SharePoint has value, but it is not a viable platform to be considered for when thinking of enterprise 2.0. SharePoint only is viable as a cog of a much larger implementation with higher costs.

It is also very clear Microsoft’s marketing is to be commended for seeding the enterprise world of the value of social software platform in the enterprise and the real value it can bring. Ironically, or maybe true to form, Microsoft’s product does not live up to their marketing, but it has helped to greatly enhance the marketplace for products that actually do live up to the hype and deliver even more value.