Search within OpenWetWare needs to be improved.

We’ve made some changes of late but there’s more to be done. If anyone has specific requests for search improvement, let us know.

We’re adding a few features to allow users to get information to us about your specific requests. Feel free to let me know what you’re particularly interested in seeing fixed.

Thanks!

Posted: March 24th, 2008 under OpenWetWare.
Comments: 1

Wascally wabbit. MediaWiki can be “coaxed” to do a lot.

The developer community spawned by MediaWiki continues to deliver new ways to extend the application in a lot of different directions. The community resembles the one that has grown up around OS Commerce. Both MediaWiki and OS Commerce are PHP-based applications with scores of implementations. Neither of them were intended to be general purpose development platforms. Both represent a specific need that a more general website may incorporate rather than doing it in a more proprietary way. Where OS Commerce has become the center of many web consultancies, MediaWiki has become the real core of many sites.

OpenWetWare is a good example. We tend to currently define ourselves as a ‘Wiki’. The collaboration afforded by the data model in many ways sometimes becomes an end in itself. Fortunately, MediaWiki is malleable enough to allow a good deal of growth as many sites change from being a Wiki to becoming more of a general web site for a specific class of content or collaboration.

One technical difference between OS Commerce and MediaWiki is the way OS Commerce extensions don’t differentiate between extension modules that require the core code to be patched and those that do. In MediaWiki, it’s more-or-less assumed that if a specific internal capability needs to be exposed, the core class method within MediaWiki within which the capability is implemented can be added to the list of internal ‘hooks’, pending review by the core MediaWiki developers. This means that the core MediaWiki class method in question will seamlessly call into a developer-defined PHP method or function do it’s thing, then post a return code that will modify the way the core class method internally handles the result.

An example of this is the way the personal navigator links appear in the top-right corner of all OpenWetWare pages. If a user enables the “Lab Notebook” option inside of the preferences section, the list of options will have a new “Lab Notebook” link added to the end. We’re soon augmenting this to allow a member to join an IGEM team and have the an icon that immediately connects them to their team’s lab notebook. No core development is done. We don’t risk complicated patches to the next MediaWiki revision to make sure the link keeps on working. It’s clean.

We use these “hooks” in several other places. They’re pervasive in the way they change MediaWiki’s behavior. Unlike the MediaWiki API, a hook can change the way MediaWiki works at a very deep level. We can implement checks on every page create/read/update/delete opertation. We can monitor how pages are changed. We can synchronize these changes with external applications.

Steps like the formal support of these hooks by MediaWiki makes the use of the application for more things than it was originally intended tantalizing. I worked on Lotus Notes (now IBM’s product) for several years. Ray Ozzie, the lead developer of Lotus Notes and now Bill Gate’s heir apparent as Chief Architect for Microsoft, implemented a similar feature in Notes in 1993. The reason was to allow security vendors to implement their own proprietary login extensions that would allow Notes to “play nice” with other enterprise applications already using their products. RSA was one of the first companies to step forward to use the functions.

What we found was that the most technologically astute and experienced developers took advantage of these hooks. The rest of the marketplace continued to clamor for a radically simpler way to interact with Notes data and processes. As a result, a very small percentage of all third-party and enterprise applications never benefited from the very deep API. There are too many reasons this is the case to go into here.

Hooking applications like this can be compared to modifying hardware interrupt handlers. The consequences of doing the wrong thing can be the infamous “white wiki page”. Granted, this can also be caused by sloppy writing on any MediaWiki extension, but it’s much more likely to happen when using hooks.

The OS Commerce extension module-writing community handles some of these concerns another way. To avoid having users edit PHP code, a version of OS Commerce, OS Commerce Loaded, was created. In this version, many of the most frequently requested extension modules were added to the core product. Clients simply deploy “Loaded” and only enable the modules they’re interested in. The core OS Commerce changes are still done, but by the community.

Where am I going with this? MediaWiki holds out the possibility to developers of infinite extensibility. But, because of how it’s done, this really hasn’t proven to be practical. Are there things that can be done with MediaWiki that may not be practical to do? It might be more useful to consider how MediaWiki can be integrated into a framework or methodology that allows it to continue to grow in a sustainable manner.

Render from MediaWikia that which is relevant only to the Wiki.

Posted: March 24th, 2008 under OpenWetWare.
Comments: none

It’s hard to get really juiced about a new application these days. So many people are doing so much to connect us to our data, shared or otherwise, in new and different ways, that it’s hard to decide which to even investigate. However, it’s nice to find an app or service that can genuinely add value.

Enter EverNote. So? I was wrong.

I installed the EverNote 2.0 beta upon the advice of Ricardo Vidal, a totally plugged-in friend from Portugal working with us on OpenWetWare as an intern at MIT for the next 6 months. The client was available for Windows, Mac OS, and Windows Mobile clients. EverNote invites you to download clients for any platform you intend to use. As it turned out, I snuck into a 12-hour free sign-up window.

EverNote provides a reasonably intelligent web-based notebook and clipboard. I used Google’s shared browser config service at one point to get at least a few of the features EverNote provides, but gave up on it; synchronizing all of my browser configurations was not as easy to use as I had hoped.

EverNote allows me to copy text or documents to my web ‘notebook’. That’s nice. Not great but nice. It also allows me to do screen captures with the same ease of use as ‘Capture Me’. But, rather than having my screenshot on my clipboard, the image is immediately copied to my notebook on the EverNote server. The client pops up showing me the new entry as soon as I select and save the clip. I can then drag the clip to the desktop. That’s a few keystrokes saved and even better.

The next level of development dazzelry is what makes it amazingly useful. EverNote does an OCR scan of every bitmap and full-text indexes the contents. The OCR failed on the first document I tried it with, a photo of a blackboard with my admittedly illegible scribbles. I tried a second screenshot, this one fled with clearly legible text. It seemed to fail once more.

But, it didn’t fail. The OCR engine took a bit of time to get around to my bitmap but soon had made every word of the document part of the text index. For the first time, it’s almost as easy to grab a bitmap as it is to copy text to the clipboard and save it that way.

For communications with folks asking for assistance, I can simply grab a screen and send it to them. The text is preserved on the server in my notebook making it accessible if I need to do the same thing again. Seeing the exact screen image is a much better way to show how to do something than talking about it. For me at least!

It’s only been a few days. For EverNote to become part of my essential toolkit, it will have to still be useful a month from now. I’ll check back then; let’s see if it cuts the mustard.

For now, EverNote is on the same level as VMWare Fusion and m trusty old “textpad.exe” text editor. I use a mac but I can still do things with TextPad that Textmate on the Mac still can’t do.

For now, EverNote is in the top 50 with a bullet on it’s way up the chart.

Posted: March 23rd, 2008 under OpenWetWare.
Comments: none

There’s only so much time in a month, especially when there are 2-3 holidays in it. I’ll do my best to keep updating the individual projects on a regular basis. If you have specific features that you’re looking for that aren’t being worked upon, let me or anyone on the OWW Board of Directors know about it and we’ll do our best to accommodate your request. It may not be immediate but I assure you that we’ll do our best to stay on top of all of them.
People

Julius Lucks generously donated his time this month to work with us on a number of development tasks. We’ve implemented a way for his project, a new Atom-based API for Cornell’s arXiv archive ( http://www.arxiv.org), to be displayable within OpenWetWare. We’ll move it to the main server soon. Julius has also been very helpful in outlining ways in which we can start implementing ways to start creating more formal ways in which individuals and their labs can be connected together.

Steve Koch (http://openwetware.org/wiki/User:Skoch3) continues to help us develop Lab Notebooks. We are in the middle of implementing prototypes for various features for the notebooks. He’s sent us a number of requests for email-enabling wikis that would help both the notebooks and a multitude of other aspects of OpenWetWare. We’ll do our best to get inbound email up and running. We’re constrained by the need to finish off other parts of the system before moving on to this one. I haven’t forgotten it, however!

Cameron Neylon (http://openwetware.org/wiki/User:Cameron_Neylon) visited with us a few weeks ago. He demonstrated some of the work he’s been doing on lab notebooks. His work is extremely interesting. He’s spent a long time working through the line between structured and unstructured applications for supporting lab science. We look forward to continuing to collaborate with him as we move forward.

We also met with Jim Hu from Texas A&M. For those not familiar with his project, e-Coli Wiki (http://www.ecoliwiki.com), check out his site. We have a lot to learn from him in the way he has built up his site. He’s also walking the line between a classical “formal” application for curating genomic information related to e-coli and providing a more fr–form wiki-based system for allowing others to co-curate. His site features a very nice integration with gbrowser (http://www.gmod.org/wiki/index.php/GBrowse), among many other tools. We look forward to finding ways to work with his site.

Here at MIT, Ilya Sytchev (http://www.openwetware.org/wiki/Ilya_Sytchev) continues to amaze me with his ability to predict the future. He told me to wait at least a few months before upgrading to Leopard on my Mac. It’s not a bad upgrade. The problem is the amount of time one needs to spend fixing problems when so few others have had to contend with them already. Installing developer tools on Friday really can make the rest of the weekend disappear. The moral is never to lse your ‘tar’ executable. And never doubt Ilya! Ilya’s keen eye can find a problem in OWW from 300 yards. My continued thanks to him for his vigilance.

Magic Links and Printable Labels

I’m still in the process of moving the “magic links” to the main server as well as providing label printing support. I’ll keep you posted on both of these topics. The mention of using a bar-code scanner generated a good deal of email traffic. I’m still working on the implementation of the label printing. I created a new extension to handle the extended “Magic Links” ( http://www.mediawiki.org/wiki/Markup_spec/BNF/Magic_links). We chose to wait until we move to the next release of MediaWiki to roll it out on the main OpenWetWare server. The original version was implemented directly in the OWW MediaWiki core and was intended for test. Expect to see it on the main system soon.

GUI Editor

A GUI editor, based on the Javascript-based FCKEditor HTML editor http://www.fckeditor.net/, was experimented with this month. There are a LOT of details still to be handled before this meets the needs of everyone. In the short term, I have it deployed on an OWW copy. Our initial enthusiasm for it was quelled by some of the more daunting usability and stability problems present with it. But we’re still working with the FCKEditor team to see how much it will take to get to the point where it’s no longer “not quite ready for prime time”. It’s a great project.

November Milestones

Although we’re still centered upon improving lab notebooks, there are several other items we’re working on. I was tardy on posting the November update but it can be found here:

http://www.openwetware.org/wiki/OpenWetWare:Software/Milestones/November_2007

This is where you can find all of the monthly status/milestone summaries:

http://www.openwetware.org/wiki/OpenWetWare:Software/Milestones

Extensions

I’ve also cleaned up documentation on extensions. All of the extensions are now linked into the main Extensions page which can be found here:

http://www.openwetware.org/wiki/OpenWetWare:Software/Extensions

For each extension, there’s currently a main page. My goal is to write and document lab-centric examples for each extension. Anyone who wants to add these should feel free to do so. Currently, extension pages are named as follows

http://www.openwetware.org/wiki/OpenWetWare:Software/Extensions/
<Extension name or Type>

Demos for each extension will also be present. They will be located directly below the name/type in a document, Demo. As an example, the full path of the RSS extension (XFeeds) demo document is

http://www.openwetware.org/wiki/OpenWetWare:Software/Extensions/RSS/Demo

If you don’t see a demo for an extension and would like to add to the docs or create a document for it, please feel free to do so.

All active extension pages are now connected together as a single category. The category page is located here:

http://www.openwetware.org/wiki/Category:Extensions

I’ll eventually add additional categories to improve discovery. I’ve imported the original doc pages for some of the non-OWW-developed extensions so people can find them. It may be more appropriate to link to some of these pages in general, but I wanted to allow the text to be searchable from within OpenWetWare.

One other note. You can also find out about all of the current OWW extensions by checking the Special:Version page here:

http://www.openwetware.org/wiki/Special:Version

This is the “official” source of what is and is not running on the server. This is a list of all currently active extensions published by the MediaWiki server. if it’s not there, it ain’t runnin!

Semantic Wiki

We’re just starting to work with Semantic Wiki. This isn’t just an extension but a way of ordering all of the information contained within OWW. The jury is still out as to how we’ll use it. I’ll keep all of you posted as we figure out exactly where it fits in. For more information, check out the following:

http://www.semwiki.org/
“OpenWetWare Developers” blog

This and other OpenWetWare development messages will be posted inside OpenWetWare as well as on the new “OpenWetWare Developers” blog. I intend to publish there on a regular basis. it can be found here:

http://blog.openwetware.org/developers

Thanks and Happy Thanksgiving.

Bill Flanagan.

Posted: November 19th, 2007 under OpenWetWare.
Comments: none