OOXML from the open source perspective

So then Microsoft decided they were going to do an open standard too, and guess what: it is a .zip file and it's got XML streams inside it. But, having said that, it has been difficult in the past to do binary file format interoperability. You can make many good arguments that it is not a benefit to have one company totally dominating the market. You need some sort of file format interoperability.

and

Isn't one file format (such as ODF) better than two? Surely the weakness of having many is the confusion it creates?
Well, yes, and it should be ODF. In an ideal world... yes, a single file format that was a superset of features and so on would be ideal, but it is very difficult to even conceive of that happening. There is just such a lot of vested business interest in this sphere. It is just very difficult to do anything technical. I just can't see anything like that happening.

Michael Meeks (part of the OpenOffice.org team at Novell)

in conversation with ZDNet.co.uk

ODT and DOCX - are they human readable?

[Note: a revised version of this post can now be found on techwhimsy.com]

One of the supposed benefits of XML is that documents produced in this format are able to be opened as a text file and read by normal people, allowing the content to be recovered, even if the formatting was unavailable After discussing the various merits of the Open Document Format (ODF) and Office Open XML (OOXML) formats (click here to read the earlier post), I was left wondering just how human readable either format was.

I created a simple document in both Open Office as .odt (Open Document Text) and in MS Office 2007 as .docx that had a heading, some paragraphs, an unordered list and an ordered list. I used the Loren Ipsum generator that can be found at Lipsum.

(click on images for larger versions)











.odt is on the left and .docx on the right

To start off with, I opened both documents up in Wordpad to see what they looked like. Not at all human readable.

.odt on the left and .docx on the right

A quick trawl through a Google search revealed that .odt is a container format that compresses all the relevant file parts in to one file. I changed the file extension from .odt to .zip and opened it up to have a look.

What worked for one format might work for the other. I took a punt, changed the file extension from .docx to .zip, held my breath, crossed my fingers, closed my eyes and double-clicked...








...and discovered that in .docx, the goodies are there, albeit buried a little deeper.

Both .odt and .docx are human readable, after a fashion. If for some reason in the distant (or not-so-distant) future either format is unreadable in its container form, with some effort the data could be extracted. It may even be possible to extract large parts of the formatting, but that's beyond my ability to assess.

In my assessment, .odt comes out ahead slightly in the human readable stakes: it isn't buried quite so deep and comes with less additional XML-related formatting and overhead. As to which is the better format overall, I'll leave that as an exercise for the reader (although I wish I could create .odt inside of Office 2007 - I do love the new Office user interface).

Open Warfare: OOXML stumbles while ODF continues to rise

Google Trends ODF vs OOXML for 2007

While Microsoft Office is the undisputed king of the office suite marketplace, the open source OpenOffice.org (OOo) has been worrying at its heels for some time. In recent months controversy has arisen over the accessibility of the XML-based file formats of the competing products - OOo's Open Document Format (ODF) and Microsofts Office Open XML (OOXML).

ODF was accepted as an international standard (click here for my earlier post on the issue) by ISO back in late 2006, giving it much needed credibity as a leading open format for documents. OOXML has also sought ISO approval but was unsuccessful in its attempt earlier this month, amidst suspicion of questionable activities of Microsoft representatives.

Why are open formats necessary?

Open formats perform an important function in the preservation of the information in documents, particularly for archival purposes. An archive is useless if it is stored in a file format that nothing read in 100 or 50 or even 20 years time. Readability is especially important for the storage of public records where there is a need for government activities to be publicly accessible in future years to future generations.

Although Microsoft's .doc Word format is nearly ubiquitous, it is far from a perfect solution. It is not uncommon for the format to become broken, unreadable and not backwards compatible between major releases of Office.

Is this the end for Office?

Defeating Microsoft should not be the main focus for OOo and the ODF, although clawing back some market share is an admirable goal and a worthy one to strive for. the user interface for Office is still a long way ahead of the its open source alternative, and in my opinion the gulf between the two has become wider with the revamped interface used in Office 2007. The differences between the two interfaces reflects the benefits that the support of a large corporation backed by massive reserves of cash and talent can bring.

ODF vs OOXML should not be an ideological battle between free and libre open source software and Microsoft. The best outcome for users is for ODF to be accepted by Microsoft as the international standard that it is and be introduced as a file export option within Office itself. Such an outcome would enable users to enjoy the best of both worlds - an excellent and time-tested user interface that also enables them to produce documents in an open and future-proof file format.

It would be a win-win situation for all consumers and ultimately, isn't that what this should be all about?

Researcher: Optimal copyright term is 14 years

To my knowledge, copyright is currently for the life of the creator plus 70 years (it used to be 50 years here in Australia until we signed the (un)Free Trade Agreement with the US who proceeded to export their copyright laws as well as DMCA equivalents to us post haste). Originally, copyright was designed to be for only a limited time, just long enough for the creator to realise some financial benefit before the work returned to the public.

Once corporations got their hands on, and in the ears of, lawmakers, this basic copyright protection has been extended and extended to where it has reached a point of ludicrousness. What possible good does it do to a society to have creative works locked away from other creative people for so long?

But I'm rambling. Lawrence Lessig is much better at explaining copyright issues than I could ever hope to do so, so venture here for a better idea of what I'm babbling on about.

I saw on Ars Technica today that an economist has determined that the optimal copyright term is 14 years. The fact that it roughly matches the originally intention of copyright law makes me a little suspicious that it has been the result of backwards reasoning - come to a conclusion and then work out the reasoning later. However, I find it difficult to fault the arguments he makes (those parts I understand at least).

Rather than have me butcher it, I encourage you to read it for yourself [PDF] and draw your own conclusions.

The Comprehensive Knowledge Archive Network

CKAN is the place to search for open knowledge resources as well as register your own – be that a set of Shakespeare's works, a global population density database, the voting records of MPs, or 30 years of US patents.

- Comprehensive Knowledge Archive Network

The latest project of the Open Knowledge Foundation. There isn't a lot there at the moment but I can see this growing in importance over the coming months and years.

Found via Boing Boing [permalink]

Citizen cartography

A career in cartography used to be the prerogative of well-funded adventurers — men like Rennell or Lewis and Clark — with full government backup ... Now, suddenly, mapmaking power is within the grasp of a 12-year-old.

- Wired: Google Maps is changing the way we see the world

I use Google Maps almost on a daily basis, particularly when trying to find the location of specific rural services. The ability to quickly locate remote locations is becoming invaluable to me.

In fact, a work-related mapping project popped in to my head which could be quite useful in informing some upcoming policy development and review processes. I wouldn't have been able to do this 12 months ago, yet alone two, three or five years ago.

Once again I find myself thinking that it is an amazing time to be alive right now, where geographically disparate chunks of information can be collated and visualised with a few clicks of the mouse.

Updated Creative Commons Licenses now available

The Creative Commons have updated the Creative Commons licences to version 3.0. According to the CC, there have been a number of changes, including an increased internationalisation.

The older "generic" CC licence was based on US law, and therefore was a proxy US version. International variations were based on the generic version and designed to be compatible with local laws.

With 3.0, the generic licence has become an "unported" licence and the language within is based upon a range of international conventions (more detail on that can be found here). Local licences will once again be written with local laws in mind.

While the CC blog outlines the spirit of the major changes, I have been more interested in the practicalities of the new licence, and whether or not I should re-licence blog content that is currently Attribution Non-Commercial 2.5 to Attribution Non-Commercial 3.0. Attribution Non-Commercial means do what you like with my text (I have only licensed my original text in the blog) as long as you attribute the source to me (pseudonym or real name) and the site and it is for non-commercial purposes. As part of the licence terms, I reserve my right to also licence content for commercial purposes, in which case contact me for further details.

Cracking the Code - Differences between 2.5 and 3.0

Now, I am not a lawyer but I had a crack at looking at the licences in detail. I looked at the "human readable" versions of the licences and did not notice any change at all. They appear to be identical (you can compare the Deed for 2.5 with the Deed for 3.0 for yourself). I then tried to read the legal code version and I'll admit that at this stage I discovered that my brain had been surrounded by a dense, energy sapping dark grey fog that reflected all light and whispered ominous messages of doom in my ear, warning that hope forsook all who ventured forth from this point.

Despite the best efforts of the portentous Cloud of Doom, I noticed that the legal code for 3.0 is quite different to the legal code for 2.5. That's not to say that this is necessarily a bad thing. I am just not qualified to make that judgement. The changes appear to be ones of consolidation and clarification of definitions and make more explicit "moral rights" and the collection of society royalties (if applicable) which would seem to me to be reasonable changes and a reasonable thing to expect in an updated licence.

What does this mean for me?

In the short-term, nothing much will change. The Australian CC licence is still at 2.5 as 3.0 has yet to be ported over. I am still considering is it worth it to re-licence and rely upon the ideas of international conventions, especially as international conventions are often observed only when it suits the observer? On the other hand, does it make much difference if I use a licence that is grounded in Australian law? On the third hand, does anyone but me actually care enough about the content I create to even want to licence it?

Since my answers to the above questions are "maybe", "probably not" and "not likely", it also comes down to convenience. At present Pixelated Dreams is licensed under a generic Attribution Non-Commercial 2.5 licence. I had thought it was under an Australian 2.5 licence but I must have decided against that at time. What I don't want to do is re-licence, and then re-licence again because at the end of it all, I'm lazy.

In my own personal circumstances, I don't see much benefit in moving to 3.0 yet. I'll keep an eye on how the new licences are received and how they spread and re-visit my decision in a couple of months. Until then, enjoy the ability to do what you like with my text as long as you attribute the source and it is for a non-commercial purpose. If you want to use it for commercial purposes, you know where to find me.

Funky space art moves into public domain

Don Davis has provided quite a lot of artist conception style paintings for NASA over the years, including planets, probes and future space colonies.

Don has passed a number of the pieces commissioned by NASA into the public domain and they can be found in all their glory on Don's public domain paintings page of his website (please note that any images on other pages on Don's site are NOT public domain).

There's some awesome artwork here.

Found via Boing Boing [permalink]

See also Paleo Future [permalink].

On Apple and DRM

Steve Jobs, in his recent "low key" posting on the Apple site "Thoughts on music", set the cat amongst the pigeons. After hinting that DRM doesn't work because "there are many smart people in the world, some with a lot of time on their hands, who love to discover such secrets and publish a way for everyone to get free (and stolen) music", he outlined three alternatives for the future:

1. status quo (everyone develops their own vertically integrated solution like the iTunes Store or the Zune Marketplace);
2. Apple could license FairPlay to other manufacturers so that songs bought through iTunes could be played on other players; or
3. abolish DRM and sell all music DRM-free and encoded in open and licensable formats.

The above is not exactly earth shattering. In my opinion, they are stating the obvious. However, Jobs then goes on to say that Apple would "embrace in a heartbeat" the DRM-free option if only the Big 4 record labels would let them (that's SongBMG, Universal, EMI and Warner). In other words, Jobs is saying "Don't blame me, blame them. We would gladly give you all what you want if they would just let us".

An important question to ask however is, why now? As has been reported on Ars Technica amongst many other places, Canada's Nettwerk Records and one of their major artists, Barenaked Ladies have been trying to convince Apple to let them sell their products through the iTunes Store as a DRM-free product, only to be told no. Of course, there are some side issues to this, including offering consumers a consistent experience (how many confused customers would complain that they could share some songs and not others, and why is the store broken?) but the fact remains that the opportunity has been there. Similarly, I'm sure many indie artists have tried to sell their songs DRM-free with the same result.

As these stories from ZDNet and CNet highlight, Apple is currently facing a number of anti-trust style lawsuits in Norway, Germany and France as European governments seek to "open up" the market currently dominated by Apple and iTunes. One of Apple's arguments against these sort of lawsuits is that, on average, each iPod has only 22 songs bought from the iTunes Store, meaning that the rest of the songs on an iPod have been obtained from other sources anyway (most likely ripped from CDs they already own - CDs being the major way music is distributed and is by and large DRM-free, excluding some reprehensible attempts from Sony with their root kits and EMI's abysmal Copy Control technology).

Similarly, another ZDNet editorial makes note that now is a good time for Apple. No DRM means no lawsuits in Europe. Even if DRM remains, Apple still owns the market. In fact, Apple is so dominant that it is probably the only company that could get away with taking a stance. The record industry is ripe for change, eager to break down Apple's stranglehold and take back some of the power they have ceded to Jobs and of course, DRM-free music is already available. The aforementioned Nettwerk already sells high bit-rate MP3s (192 kbps, as compared to 128kpbs AAC files from iTunes) from their website and eMusic has sold DRM-free tracks for many years with some success.

Not surprisingly, the RIAA appears to have missed the point. This LA Times article quotes the RIAA as saying:

Apple’s offer to license Fairplay to other technology companies is a welcome breakthrough and would be a real victory for fans, artists and labels. There have been many services seeking a license to the Apple DRM. This would enable the interoperability that we have been urging for a very long time.

I don't think anyone else in the world seriously thought that Jobs was offering to open up FairPlay (in fact, Jobs argues the opposite when he highlights that opening up FairPlay just increases opportunities for enterprising individuals to hack away at the DRM system). Whether the RIAA is making a point of its own or is just plain stupid remains to be seen.

The Economist sums it up best:

Mr Jobs’s argument, in short, is transparently self-serving. It also happens to be right.

More free tech goodness: FreeTechBooks and ITConversations

I don't hear about either FreeTechBooks or ITConversations very often, and I think that's a real shame. Both of these sites are an amazing resource for anyone tech-geek inclined, although possibly more so if they are like me and just a tech-geek in training. What they lack in Web 2.0 shiny goodness, they make up for with that holy grail - content.

FreeTechBooks has, as you can probably guess, free online versions of computer science and engineering books and lecture notes. While some of the texts are not spectacular, there are some seminal texts such as How to Think Like a Computer Scientist (with versions aimed at learning Python and Java) and little gems such as the first edition of Programming Ruby: The Pragmatic Programmer's Guide (from the minds of the famous Pragmatic Programmers).

ITConversations is an ever-growing collection of seminars, keynotes and conference speeches on a mammoth range of IT topics. While there's a lot there that doesn't interest me, there's some fantastic speeches from Woz, actor/writer Wil Wheaton, sci-fi author Bruce Sterling, security guru Bruce Schneier and Creative Commons founder Professor Lawrence Lessig. There's something there for everyone and it's all free.

Both sites receive the (admittedly worthless) official PixelatedDreams seal of approval.

Open Document Format published

The Open Document Format (ODF) has finally been publishedas an official standard by the International Organization for Standardization (ISO). What does this mean?

ODF is an XML-based file format that saves text, spreadsheets, databases, presentations (think PowerPoint or Keynote) and the like to a standard, open format that can be implemented by anyone who wishes to, free of charge. The importance of XML is that, by and large, XML-based files are more or less human readable. I've never tried to read a raw ODF file, but the fact that ODF is now an open published standard, it means that anytime in the future anyone who knows how to read a published standard will be able to develop a reader for ODF files.

I have no illusions that ODF will be the downfall of the "Great Satan" - of course it won't. A great many people use and value Microsoft Office and will continue to use it for many years to come. However, as it currently stands, Microsoft Office file formats are proprietary and not open. Applications such as OpenOffice.org have reverse engineered various forms of Office file formats but Microsoft could break that functionality quite easily in future releases. Having an international, open, published standard means that documents can be archived in ODF format today and still be read 20, 30 or 50 years in the future (assuming the media they are archived on is still readable). If Microsoft disappeared off the face of the Earth five years from now, the ability to read those office formats would diminish over time.

Theoretically.

I'm also aware that Microsoft has submitted their Office XML standard for publication/standardisation. ODF has simply got there first. Hopefully Microsoft will support ODF as well as their own file formats in future releases of Office.

Found via ArsTechnica [permalink]

Open source education

Prestigious Yale University in the United States has announced that Yale will make select courses available on the internet in video format from "fall 2007" (Spring for us Southern Hemisphere types, or Q3 07 for the quarterly inclined). The Yale Open Educational Resources Video Lecture Project is being supported by the William and Flora Hewlett Foundation and its Open Educational Resources project.

Similar "open source" education initiatives are already in place, most notably MIT's OpenCourseWare and Rice University's Connexions. However, the Yale program has an edge in that they will be making videos of lectures available as a core component.

While it is true that being able to access the information is only one step towards learning, and I don't think anyone could mount a strong argument that this is the beginning of the end for traditional universities, it's a positive move forward. Most information available like this has a Creative Commons or similarly liberal licence attached, making it an excellent vehicle for developing education programs in developing nations. In my mind, this is where the true potential for these programs lie - helping to raise the level of common human knowledge for the benefit of everyone.

For the more historical geek, it's further evidence that ArsDigita Univeristy was way ahead of its time.

(First spotted on InsideHigherEd.com).

Hot Chips: Open Hardware

It's becoming increasingly obvious to me that I have no idea what is going on the tech world around me.

I have only just discovered, with the launch of the Simply RISC S1 Core, that not only is there such a concept as "open hardware", but hardware designed from open sources is actually being shipped.

It is pretty cool that Sun has made the designs for their SPARC chip freely available with the OpenSPARC project. The SPARC is a complicated looking chip that appears like it has a lot in common (although very different to) the STI Cell chip (currently most well known as the powerhouse for the PS3).

What is cooler is that there is a community building up around the idea of developing an open System-On-a-Chip (SOC) infrastructure at OpenCores.org.

I'm happy to admit that understanding the potential and comprehending the possibilities are beyond my grasp, but the idea that there are freely and readily available resources out there that people can use to design and maybe even build their own CPUs or even multi-cored, integrated SOCs. This isn't really a big concept for the industry giants, but it would have to be exciting for the budding electrical engineer who has a great idea about a new system design but is not quite sure where to start. It's not just new systems that excites me. My mind boggles just trying to think about where the more creative tinkerers can get to when designing DIY and hobbyist electronics with embedded processors. That's a whole other world away from DIY printed circuit boards (which is hard core and high tech enough to scare me).

I get goose bumps just thinking about it. Now I think I understand a little of what Woz felt in those early days.

TED

I am happy to admit that I'm not as switched on or tuned in to the world of tech and information as I would like. One of the problems of loving information as much as I do, and having as little an ability to filter information that I do, is that you miss a lot of things that could well be common knowledge to so many other people. Today I am talking about TED - Technology Entertainment Design.

Apparently, every year more than 1,000 of the best and brightest in Technology, Entertainment and Design are brought together to discuss anything and everything over 4 days in Monterey, California. It's the kind of intellectual gathering on just about everything that makes me giggle in excitement at the thought of just being able to listen in on mere fragments of the conference presentations.

Over the years, many inspirational and visionary people have spoken at TED, including Jane Goodall, Al Gore, Bono and Richard Dawkins. What has really got me so excited though is that TED, in partnership with BMW and promoted by Google Video, are releasing a selection of some of the best TEDTalks.

I'm a firm believer that information wants to be free. The fact that TED have opened up their archives like this truly excites me in a way that leaves me disturbingly inarticulate and unable to express myself. I've started downloading a couple of the TEDTalks, but I'm also in the middle of mid-Semester assignment/exam mode, so I can't really spend much time on them. TEDTalks recommend that for the best effect, "plan to listen to at least three, from start to finish. They have a cumulative effect...".

There are many to choose from: Al Gore, Mena Trott, Tony Robbins, Nicholas Negroponte ... there's a long list. My own personal favourite though is Sir Ken Robinson, a world renowned speaker on creativity and education. He gave the closing address at the International Education Forum, which is how I first came across Sir Ken (I didn't attend the forum, but I have seen a DVD of his address, and it was very impressive). Sir Ken's closing address and this TEDTalk cover very similar themes, and I defy you to not be inspired by it, even just a wee bit.

I know that over the coming weeks and months I will be delving deeply into TEDTalks, trying to open myself up to the many ideas I can just feel coursing in raw pulses across the ether.

Information is such a powerful thing. I hope we can all continue to share it.