TWIKI.NET Blog Voice

TWiki User Meetup in Silicon Valley, 2008-05-16

Our third Silicon Valley TWiki User Meet-up will take place at the Plug and Play Tech Center in Sunnyvale on 2008-05-16. This is a good opportunity to mingle with wiki aficionados and to learn from each other what does/does not work when deploying a wiki at work. We hope you can join the event, and possibly present how your organization is using TWiki?

Plug and Play Chefs will prepare gourmet pizza and provide drinks for this TWiki User Meet-Up; TWIKI.NET is sponsoring this event.

If you are interested in getting notified of future TWiki Meet-ups in the Silicon Valley you can subscribe to the [twiki-users-sfbay] mailing list at http://lists.sourceforge.net/mailman/listinfo/twiki-users-sfbay.

pencil 2008-05-12 | Peter Thoeny | Category Community | Permalink

Scalability of TWiki

Sometimes we get the question on how well TWiki can scale. This blog post compiles scalability related information so that you can plan your TWiki deployment effectively.

Scaling Across Teams and Departments

TWiki was designed as an enterprise wiki from its inception. You find features specifically designed to support large deployments. Other wiki engines have a different focus and may lack some of these features. Wikis typically flourish in grassroots. Once at the radar screen of the CTO/CIO, grassroots wikis often get consolidated into a central TWiki. That is when scalability comes into play. Key scaling features of TWiki:

  • Multiple webs (workspaces):
    • You can create as many webs as you need. Some large TWiki deployments have over 1000 webs. Think of a web like a wiki within TWiki. Each team can get their own wiki. People need to register only once, then they can create content in their own space. If needed you can link across webs, such as to reference a registered user or an entry in the Glossary web.
  • Fine grained access control:
    • You can create TWikiGroups and restrict access to content for view and edit based on those groups. Although it is possible to restrict access on a topic (page) level, it is typically done on a web level for ease of administration.
  • Authentication:
    • In a large deployment it is advisable to authenticate users against your directory server, such as Active Directory or LDAP. That reduces the workload on registration/login questions.
  • File attachments:
    • TWiki has a per-topic namespace for file attachments. That means, if one team uploads a file called inventory.xls to their team page, and another team uploads a file of the same name to a different page, they will not collide. Try that with Mediawiki or other wikis.
    • You can limit the maximum size of attachments that can be uploaded. This can be done for the whole site and also on a web level.
  • Organize content:
  • Web 2.0 platform:
  • Create your own applications:
    • TWiki goes beyond Web 2.0: The TWiki platform is about user generated application logic. Your users can create situational applications that solve specific business needs, such as a bug tracker, a employee news portal, TWiki's Support web and more. You do not need to be a programmer; all application logic is done in TML (TWiki Markup Language) using TWiki forms, reports and optionally some HTML and JavaScript.
    • The IT department is in charge of the wiki dial tone and wants to have some control over the wiki deployment. With TWiki you allow users to experiment in a controlled environment. That is, IT can get the dreaded "shadow IT" under control.
  • Integrate:
    • TWiki has a plugin API and ready made plugins to connect to external databases. That way you can run a query in MySQL and other RDBMS and display the result in TWiki pages. Useful to show CRM data to sales teams and bug trends to engineering teams.

Server Selection, Caching, Load Balancing

Plan for adequate server hardware when you deploy TWiki. The following are ballpark figures for an sample TWiki deployment serving 1000 employees and 50,000 pages:

  • Enterprise class Linux
  • Dual core CPU 2.6 GHz
  • 2 GB RAM
  • RAID 1 or RAID 5 for redundancy
  • Dual power supply for redundancy
  • Plan disk space:
    • Page content: 15MB per 1000 page (yes, MB, not GB)
    • File attachments: 1GB per 1000 pages

If you have a high read to write ratio (such as a TWiki on the public internet) consider a caching solution and/or a load balanced setup.

  • Load balancing:
    • For high volume traffic sites it is possible to put TWiki on a load balanced setup. Here is an example:
      • Cisco Ace load balancer.
      • 3 webservers.
      • NAS storage back-end.
      • Webservers share data on NAS for pages, file attachments and log files.
    • In the early days, TWiki.org was on a load balanced server setup while hosted at SourceForge.net. Now it is on a single and aging server hardware. The TWiki community plans to move TWiki.org again to a load balanced server setup which will improve the performance considerably.

Scalability of Search

TWiki uses the Unix grep command to search content in real time. This enables flexible and powerful searches in real time, which is important for TWiki applications. Search is covered in related pages in the TWiki web: SearchHelp, VarSEARCH, QuerySearch, FormattedSearch, and on TWiki.org at TWiki:TWiki/SearchSupplement.

The real time search has a performance impact. Searching all webs in a TWiki sites with more than 50,000 pages can be slow. If you have a large TWiki deployment of more than 50,000 pages it is advisable to index TWiki content with a search engine. This can be done with a commercial search engine such as the Google Search Appliance or an open source search engine. TWiki currently has three open source search engine integrations: TWiki:Plugins.SearchEnginePluceneAddOn, TWiki:Plugins.SearchEngineSwishEAddOn and TWiki:Plugins.SearchEngineKinoSearchAddOn. See more extensions on search.

To scale the queries of TWikiForms based TWiki applications look into TWiki:Plugins.DBCacheContrib and TWiki:Plugins.DBCachePlugin.

Flat File Back-end

Some people express concerns that TWiki's flat file back-end does not scale well. We know of a number of large TWiki deployments that have over 300,000 pages (such as at Yahoo), over 1000 webs (such as at a major telco company), and over 10,000 users (such as at a major financial institution in USA).

A flat file based storage back-end has several advantages:

  • Simple installation
  • Simple backup and restore
  • Simple migration of content between TWiki installations (think of grassroots wiki consolidations, spin-offs and acquisitions of companies)
  • Well understood caching and replication technologies available
  • Resilient to data corruption

Some scaling factors:

  • TWiki scales well on number of webs, e.g. it does not matter much if you have 3 webs or 3000 webs.
  • TWiki has a limit on the number of pages in a web. You will see a performance impact if you have more than 20,000 pages in a single web. This depends on the file system/configuration used, on the bandwidth of your server I/O and on the memory installed.
  • TWiki scales well on the number of registered users. We have not done tests on the upper limit. It is also feasible to not register users in TWiki, e.g. to rely solely on LDAP login.

As stated above, performance can be addressed with caching and/or load balancing.

The TWiki community is working on a pluggable storage back-end, see TWiki:Codev.TWikiRoadMap.

pencil 2008-03-25 | Peter Thoeny | Category Best Practices | Permalink

New Leadership Supporting Community

Dear TWiki open source community:

If you google the news on "twiki" you will see a lot of press coverage today.

Rod Beckstrom accepted a high level position within the US government. He was appointed to run the newly created National Cyber Security Center, which will operate within the Department of Homeland Security. I can't express in words how much I appreciate Rod's engagement with the company and with the community. A true leader.

I am thrilled to announce that we appointed Tom Barton as the new Chairman and interim CEO of TWIKI.NET. He is very much execution focused and has an excellent track record in developing vibrant open source communities. Tom was the Interim CEO of Cygnus Solutions, which was acquired by Red Hat Software in 1999. At Red Hat, he continued as Senior VP of Client Services. More recently, he was CEO at Rackable Systems, the company he made public.

Tom asked me to convey this message to the community:

I am very excited about joining TWIKI.NET and the TWiki community more broadly. We used TWiki extensively at my last company - Rackable Systems - and I saw the difference it made in our efficiency by improving departmental and cross functional collaboration, not just as a tool use by engineers to manage projects. TWiki was used throughout the organization. It virtually replaced our traditional IT managed intranet. For example, our TWiki based intranet contained all HR information, our quality system, manufacturing instructions and other business critical information.

My experience was that the software is extremely reliable. In the 4 years I was with the company, the only time the server failed was due to a hardware fault, so obviously the team has done some excellent coding. It is clearly the best enterprise wiki out there and has the most active and vibrant open source community. I am committed to making TWiki.org a success as much as I am committed to making TWIKI.NET a success. When I was at Cygnus Solutions, we had a very positive relationships with the GNU project, and through our efforts as a commercial company, we were able to fund a substantial portion of the development of core technologies like gcc, gdb, and glibc. I hope to help do the same with TWiki.

I asked Tom to participate in our upcoming open source release meeting on 31 Mar 2008. He can't participate this coming Monday since he will be out of the office all next week.

With Tom on board we will continue with the mission I stated last year on twiki.org. I believe TWIKI.NET executed well so far along the commitments:

  • Fund enhancements to open source project via consultants (editors, skins)
  • Organize and sponsor TWiki Community Summits (including Larry Wall)
  • Organize and sponsor TWiki Meet-ups
  • Secured expensive Sun hardware for twiki.org
  • Fund half a rack of hosting for twiki.org
  • Help publicize TWiki (such as TWiki's position in Gartner's magic quadrant)
  • Initiated TWiki merchandise (T-shirts)
  • Refer consulting work to consultants

Please let me know if you have suggestions on how TWIKI.NET can bring more value to the TWiki community in a most effective way. Please provide feedback on the TWIKI.NET page on TWiki.org.

pencil 2008-03-20 | Peter Thoeny | Category Community | Permalink

TWIKI.NET Sponsors YouTube Contest 2008

YouTube is the second most popular website in the world and every day over 10 million videos are watched on the site. It is a great platform to promote products and TWIKI.NET plans to tap into this to build the awareness of TWiki. All we need are some compelling videos!

The expertise to make these videos is out there in the TWiki community and every TWiki user we meet has their story to tell. All people need is a gentle push to get those cameras running and start posting TWiki videos on YouTube.

TWIKI.NET are sponsoring that gentle push to the tune of over $3,000 dollars with the TWiki YouTube video contest. We are awarding prizes for the top three as well as giving every qualifying video entrant a free T-shirt.

  • 1st Prize - $2,000
  • 2nd Prize - $750
  • 3rd Prize - $500

Find out how to enter by going to the TWiki community contest page on the TWiki.org website. Entry is free. All you need is a video camera, some ideas and you could be taking home $2,000.

pencil 2008-02-18 | Michael Corbett | Category Community | Permalink

Videos of TWiki Meetup 2007-11-29

TWIKI.NET organized the first Silicon Valley TWiki User Meetup at Plug and Play Tech Center on 29 Nov 2007, where more than 50 TWiki users participated in the event. We would like to share some videos.

TWiki Meetup introduction

Rod Beckstrom speaks on collaboration

Vicki Brown discusses how Yahoo! uses TWiki

Guy Martin presents Motorola's TWiki

pencil 2008-01-28 | Amir Shobeiri | Category Community | Permalink

TWiki Meet Up - Silicon Valley - 29 Nov 2007

We had a wonderful first Silicon Valley TWiki User's Meet Up at Plug and Play Tech Center Thursday night (Nov 29). More than 50 TWiki users were in attendance and highlights from the evening included a superb presentation by Vicki Brown of Yahoo! and Guy Martin of Motorola Open System Technology, among others.

Vicki Brown explained how Yahoo!'s TWiki site had grown to incredible richness with 318,000 live pages, more than 10,000 users, 5 MILLION page views per month and more than 500,000 TWiki page edits per month. This is perhaps one of the largest and most successful wiki and collaboration platform implementations in the world. She showed some of the TWiki programming "magic" she had developed to help manage the growth and evolution of the system.

Guy Martin declared that Vicki was his hero and explained how he and his group were supporting and disseminating TWiki on different projects in Motorola. He also discussed how he wanted to see more interfaces developed to products like De.lic.ious and Scuttle, only to learn that another company in attendance is already working on such a new TWiki plug in.

Jason Sares of Giant Dorks talked about two brand new TWiki installations he did for two Silicon Valley tech companies.

Presentations were also made by Ian Kluft, and Jack Button

Peter Thoeny, the founder of the TWiki open source platform and Co-founder of TWIKI.NET gave an overview of the technical foundation of enterprise, structured wikis in contrast to the many publishing wikis that exist, and gave some programming tips on how to create more powerful collaborative applications using TWiki.

Rod Beckstrom, Co-Founder of TWIKI.NET and Co-author of "The Starfish and The Spider: The Unstoppable Power of Leaderless Organizations," presented the concepts from the book and how they apply to enterprise collaboration. He also gave a summary of TWIKI.NET’s business strategy and offerings.

In the end, prizes were given and were won by Vicki (an Apple iPod), Guy (Rod's The Starfish and The Spider…book and Ian (Peter Thoeny's Wikis for Dummies Book).

Many thanks to all who attended and contributed.

Meet up, Rod speaking
meetup2.JPG

1st Place -- TWiki Champion Vicki Brown from Yahoo!, with her prize, a New iPod Nano.
VickiBrown.JPG

2nd Place -- TWiki Champion Guy Martin from Motorola, with his prize, "Wikis for Dummies," co-authored by the father of TWiki, Peter Thoeny
GuyBook.JPG

Jason Sares from Giant Dorks and Rod Beckstrom
JasonRod.JPG

pencil 2007-12-01 | Rod Beckstrom | Category Community | Permalink

Case Study: KQED's QUEST Program Managed by TWiki

Last night I went to the Wiki Wednesdays' October event in San Francisco. KQED, the National Public Radio/TV for Northern California, gave a presention on how they use TWiki in a structured way to manage the QUEST program. "QUEST is a TV, radio, web, and education series by KQED that explores science, environment and nature in Northern California." The program runs for 3 years and will produce 60 TV features and 48 radio features per year, as well as daily blogs and photo albums on Flickr.

Craig Rosa, the interactive producer of QUEST, and Lauren Sommer, the wiki champion first gave an overview of QUEST, and then explained how their structured wiki approach solved their needs. Usually, the TV folks and radio folks work without much interaction, but in QUEST they focused on getting getting all parties work together. The question was how to solve the cross-editorial challenges.

They looked at wikis listed at WikiMatrix. The criteria was:

  • Cross-platform and standards based
  • Programmable
  • Simplicity
  • Free or low cost
  • Tie to authentication system
  • Access restriction, but accessible from outside
  • Skinnable

After looking closer at TWiki, JotSpot, MediaWiki, PBwiki they decided to use TWiki, partly because it was already in use within KQED. One year ago they started with a new TWiki web for the QUEST program. Now they have 20 active users and 25 consumers of the wiki. They created several application over time to track story ideas, track approved features, and to coordinate other activities.

AJ Alfieri-Crispin, the TWiki administrator, created a TWiki Forms based application to track story ideas. A person can submit a new story idea by filling out a form: Title, description, contact person, media (TV, radio, community outreach, e.t.c.). They created the application in iterations, it was improved over time as they discovered usage patterns. The executive producer of QUEST stressed that all ideas need to be tracked in the wiki, "a story idea does not exist unless it is in the wiki". Initially, people just check-marked the media they are familiar with. Over time they interacted and learned from each other, and started to check-mark other media as well.

They defined an approval process to promote a story from an idea to an approved feature. Once approved they changed the TWiki form to the TV form (for a TV feature), a radio form (for radio), etc. Doing so pulled the story from the idea table and pushed it into the approved feature table. These tables are dynamically generated with a TWiki SEARCH. For each feature page they have supporting pages, created by clicking on a button in the feature page. One supporting page is an executive summary page that summarizes key data of each feature. This info is used to update the CMS that drives the website. For example, the TV episode Earthquakes: Breaking New Ground, lists title, description, latitude/longitude, air date, and episode number.

They use mostly TWiki to manage the QUEST program, with a few exceptions. Scripts are done in Word since they are updated very frequently by a single person. Once finalized it is stored in the wiki. Calendaring is done in Outlook, with a web interface. Their wish list for the wiki include a better WYSIWYG editor, a better way to attach large amounts of files quickly, better Office tools integration and better calendaring.

They installed the TagMePlugin for content tagging, but it is not used since there is little need for folksonomy because content is already highly structured.

Most people were new to wikis and nobody knew the structured wiki approach. The tool adoption caught on quickly because the executive producer promoted the wiki. The wiki champions got e-mail such as "can you put this into the wiki for me?" They were gently reminded to actively use the wiki. TV and radio is a fast moving environment, they plan to produce 500 stories in 3 years. They feel that it would not be possible to manage all these stories without the structured wiki approach. "The wiki is a matter of survival for us," Craig said. He estimates that what would take him one hour to do in the past takes now a few minutes.

Lauren gave a live demo of the wiki by adding a new story and promoting it to an approved feature. Pierre Khawand of People-OntheGO recorded the presentation for TWIKI.NET; we plan to provide a video summary that highlights key points.

pencil 2007-10-04 | Peter Thoeny | Category Case Studies | Permalink

TWIKI.NET launched at LinuxWorld in San Francisco

We had a very fun and busy launch of the TWIKI.NET at LinuxWorld in San Francisco, 2007-08-07. Several hundred TWiki users and fans visited the site as well as 700 other people who were interested in learning more about our powerful open-source wiki and collaboration platform. Our booth was busy the entire conference. One of the staff from LinuxWorld commented that it appeared to be one of the busiest booths at the show.

Dsc09925b.jpg

Many people were thrilled to be able to meet our co-founder Peter Thoeny, the original creator of the open source TWiki. We also met many existing and new industry friends and supporters. I had the opportunity to present at the Intel Pavilion for two showing. David Allen, from our product management team, spoke on Linux intrusion detection and security.

We also had support in our booth from business partners such as FAonDemand, who have built TWiki based applications for us and other customers, and also LignUp. LignUp provides VOIP development platforms and has produced some very cool TWiki applications such as a voice-enabled wiki for project management. The application allows any team member on a project to call a phone number and leave a message on a wiki page for the rest of the team to hear. It also automatically notifies everyone on the team by email that the message has been posted on the wiki project page. If you are interested in learning more about that application, please contact Claire Umeda at LignUp.

Many thanks to these partners, and the entire TWiki community at LinuxWorld for your support!

For more information on the launch, please visit the Beckstrom Starfish Report.

pencil 2007-08-09 | Rod Beckstrom | Category General | Permalink

TWIKI.NET Blog

Recent Posts