May 2005 archives from

Piotr's R&D blog

This is the most influential paper award presentation. Michael Jackson (from the panel) is rehashing some of the same points re: normal vs. radical design. From what I understood, normal design is based on experience and doesn't deviate too far from things that have been tried and are known to work. Radical design tries to build something new, based on theory or formal models. The best that should be expected of a radical design is that it not be completely broken, and provide the motivation for further design work. I guess that software engineering is doing mostly radical design so far, and Michael would like to see it move towards normal design. But is it possible that the nature of software precludes normal design a priori?

Pamela Zave is now talking about applying some of these principles to telecommunications. I didn't see the relationship to Michael's talk, it seemed to be about an architecture for composable telecom system features. I'm afraid I didn't quite follow, though.

The Evolution of Development at Microsoft

This talk flowed so well, I didn't want to be distracted with taking down details during the presentation. The basic premise was that Microsoft used to hack things up quickly because that's what the environment (other hackers) accepted; they just weren't bothered enough about bugs to make it worthwhile to invest into QA. The landscape has changed, and Microsoft now concentrates on quality a lot more. They are switching to agile practices, but each team (of 400+ total) chooses their own process. They do have common quality gates (e.g. testing coverage, coding standards, published specs, etc.) that all the groups have to pass through. The speaker encouraged people do "just do what makes sense": you need some rules to avoid chaos, but not so much as to stifle creativity and motivation. Mostly, it comes down to hiring good, ambitious people, and having them compete against each other.

Software Architecture in an Open Source World

Roy Fielding gave this talk, but while I admire the guy's work and his no-nonsense (if a little adiplomatic) attitude on mailing lists, the presentation was pretty bad. He spent most of his time talking about seemingly meaningless details of the various software systems he's been involved with. It's entirely possible that those details were meant to be examples of architectural principles, but if so he hid the relationship really well. Yawn.

What a contrast to Erich's talk... The speaker is apparently a marketing guy serving a warmed-over business presentation with generous servings of rethoric and argument by assertion ("you know it's true!"). His argument boils down to: naughty developers don't follow the architectural models handed down from on-high, so we must automate the transformation from model to code (...and fire the programmers?). To be able to automate anything, machines must be able to understand the models, so clearly purely visual diagrams that only communicate to humans are inadequate. (This reminds me of the Web vs. Semantic Web debate: the web is only human-readable, while the semantic web makes all that information computer-understandable. Now, which one is wildly successful and which one can't seem to take off even with Tim Berner Lee's personal backing? One guess only!)

The speaker has finally gotten around to talking about MDA proper. It's the usual PIM to PSM to code flow chart, but at least he freely admits that the PSM will need lots of hand-tweaking to actually work. But then he goes right back and claims platform-independence, longevity, filling in pattern templates, etc. Blah. Interestingly, a guy from Motorola stood up and claimed that they're doing MDA successfully, precisely as presented. Gotta look into this to figure out what kinds of applications it's appropriate for.

This speaker promises to explain how his organization migrated from a plan-driven (waterfall) development process to an agile one. He starts out by defining plan-driven and agile processes, including a good analogy: plan driven is like building a bridge, agile is like exploring where and how to cross a river. The difference can also be characterized by the tradeoff between YAGNI and DOGBITE: You Ain't Gonna Need It vs. Do it Or you'll Get Bitten In The End.

There follow lots of details about how to convince stakeholders to accept an agile process, and how to set up the process to have the best chance of success. I won't bother scribing all this stuff; if the slides aren't posted eventually on the ICSE site, you can probably get most of the same information from books.

Lessons learned:

1. The strongest motivator by far to switch to agile development is failure in previous projects.
2. Expect resistance from Software Engineering Process Groups (in large organizations). (A solution: rotate staffing of SEPG.)
3. The traditional procurement approach -- write requirements specs, invite for tender, select cheapest offer -- does not work for agile development. (Anecdote: WTO contracts prohibit requirement writers from bidding on the project, and limit with bidders!)
4. Completely discarding change request processes would be like throwing out the baby with the bath water. (Suggesting to have a lightweight feature request process to avoid duplication and repetition.)

Erich will be talking about agile, open source, distributed and on-time (and buzzword-compliant, apparently): inside the Eclipse development process. Unfortunately, his accent is difficult to understand over the microphone, and the font on his slides is barely readable [1]. Oh well.

In Eclipse development, the emphasis in on people over process, and on shipping software over other activities, producing a culture of "If you ship, then you may speak". They ship regularly, each milestone is a miniature development cycle so they get feedback early and often. Within a milestone, continuous integration is important (nightly, weekly and milestone builds), and of course they eat their own dog food daily. They also encourage and depend on community involvement, to avoid developing in a vacuum. This requires extra work and support on their part: for example, they weren't getting much feedback on milestone builds, because people didn't know what was new in each one, so they started publishing "new and noteworthy" lists and now get much better results.

Unsurprisingly, they also emphasize testing; the goal is to reduce the time between a bug appearing and being detected. They also have automated performance and resource (?) tests. In their release endgame, they alternate testing and fixing. They do not separate testing and fixing responsibilities: everybody does everything. After the release, they run a decompression cycle and retrospective. The timing is: 9 months of milestones, 1-2 months of endgame, 1 month of decompression.

The plan is prepared early, but remains alive until the day of the release. Risks are mitigated by putting high-risk stuff up front, and dropping things rather than pushing out the schedule. Everybody shares ownership of the code to ease resolving inter-component issues. Teams are put together dynamically to handle tough integration issues.

Eclipse is built to last, with an analogy to actual buildings. A building has multiple layers: the geographical location (permanent), structure (30-300 years), services (plumbing, electricity, 7-15 years) and stuff (furniture, replaceable easily). The various layers change at different rates, so the building's design must tolerate shear between the layers. This is painful but necessary if you don't want the building to become obsolete. In Eclipse, the structure is the plugin layer, the services plumbing are the APIs, the stuff is the UI. APIs matter, and are kept minimal and backwards compatible as much as possible. APIs are designed at the same time as the implementation and the client, to make sure they are practical, and serve to isolate components to maintain development velocity within each component (standard modularity argument).

Things don't always work out. Lower-layer ripples sometimes have a big effect on upper layers; as a result, they have a rule that a lower-layer change is not done until the ripple has (successfully) propagated all the way up. Dynamic teams are not always successful, and require lots of leadership and management support. Finally, they do drop features -- perhaps this could be seen as a gambit to avoid even worse things happening?

Footnotes

[1] Note to self: self, don't use fonts with thin lines, as they'll get washed out on the projector.

P.S. Why is it that today I see people coming in with breakfast goodies at 9:45am, but on Wednesday, when I was 5 minutes late, the breakfast had been taken away by 9am sharp? There is no justice in this world.

This is a long and rambling entry -- panels are very difficult to summarize, especially live, on the spot. Feel free to ignore this item, there's nothing groundbreaking in here.

There's 3 panelists, and they gave 3 very different position statements; different not in that they have different opinions on the topic, but in that they weren't even talking about the same thing! They all seem to argue, though, that we need science in design, whatever it may be, and that we should reach out to other disciplines for help. Humph. I have nothing against interdisciplinary approaches, but why don't we figure out what our problem is first? Otherwise, I think we run a high risk of "discovering" our problem is a nail because other disciplines treat their problems with a hammer.

OK, so we don't know what the science of design might be. How will we know when we achieve it? One suggestion is that design will become a science when it stabilizes, similar to the stabilization of mechanical designs after an initial period of radical experimentation. The counter-argument is that anything that's stabilized is science, but is no longer design -- design will move on to stay at the edge in order to continue creating value. Mary Shaw also points out that abstraction, in the form of evolving programming languages, is a form of standardization of design. Carliss argues that value does not go away with standardization, rather it becomes difficult to capture, which is good for the majority since value then flows to all of us. Beyond standardization, how can we harness creativity on a regular basis, without stifling it with templates and formalizations?

Science is about "compressing" information -- finding small representations for big classes of complex behaviours -- while keeping these representations practical for the problems at hand (not "first principles"). Science is not the same as engineering, but according to this guy the panel isn't making a good distinction. What is the difference, especially in the context of software?

There's a difference between "normal design" and "radical design", need to look it up. Apparently, we teach normal design well, but fail on radical design -- which is unfortunate, because that's the really difficult stuff.

Software is malleable, so we shouldn't have a single representation for design?! Mary says you should create notations specific to your problem (DSLs?), but admits that interoperation between notations would be a big problem. Seems to me that this problem has been solved; most people can use UML, or if they prefer can extend it, or build a new language on top of the MOF metamodel that will still retain some basic compatibility with other representations based on MOF.

And here we go with analogies to other fields again... What's the equivalent of a wind tunnel for software? Who cares? Mary would have us look at civil and chemical engineering, Carliss says system engineering (but that's 60% software already), and Michael says we should just look at other branches of engineering in general. Yawn...

Mary would like to predict quality attributes based on a design, sounds like ABAS.

Conway's Law: a design reflects the structure of the organization that produced it. Carliss thinks we should formalize this hypothesis and test it. Sounds fun, though I'm not sure it would be useful.

Two great points from an attendee: first, we need to standardize a vocabulary, second, design is a skill not knowledge.

This morning, after watching the rather good Lewis & Clark movie at the arch, I came back to get an informal demo of the Continuous Testing plugin for Eclipse. The concept is really simple: it does for JUnit tests what the Eclipse compiler did for Java source code. Tests are automatically executed in the background as code changes, and errors reported in the standard Eclipse problems pane. You can offload running the tests to another box if your development machine doesn't have enough oomph to handle it comfortably while you're coding. I'll be installing it as soon as I get home, and if you write any Java in Eclipse, so should you!

I'm in an invited talks session with a very generic title. The invited talks get the biggest room at the conference, and seem to be well attended.

Usability in Open Source

Thesis: open source deals well with functionality challenges (by distributing the work, mainly), but is not so good at usability. We're trying, design blogs are good, but need to do more to involve users. The speaker was good, but the presentation went all over the place at extreme speeds, so it's very difficult to summarize. His final idea is the good old "let's make putting applications together easier so we don't need so many software engineers". Of course, he doesn't say how to do it.

How software can help or hinder decision making

A real live psychologist is presenting! He looks the part, with wild grey hair and a hunched, understated look. Unfortunately, his slides are alternatively full of swaths of text and numbers, occasionally interrupted by a series of all-too-similar graphs. he goes into way too much detail on his study, where he seems to have demonstrated that a computer system that claims to assist in mammograph interpretation decisions can actually harm them. Although studies show that on average the system works fine, he uses some fancy statistics to reveal that its effects depend on the skill of the user (I think). Overall, I think he failed to aim his talk properly at his audience, since I still have no idea how this affects software engineering beyond "computers can hurt people, be careful and consult a psychologist who knows statistics". Oh, and learn how to prepare talks for a non-specialist audience.

I stopped by for an informal demo of Ossher's Concern Manipulation Environment. Ossher started off the whole aspect movement before he got eclipsed by AspectJ. He's still working on HyperJ and CME. I think his ideas are more general and powerful than those of AspectJ, but he hasn't been successful about implementing and popularizing them so far.

Anyway, on to CME. It's yet another Eclipse plugin. It slurps in info from many sources (main Java code for now, but extensible) into a neutral model. You can then query over the model and materialize queries into "concerns". You can also do some kind of composition between concerns, but that wasn't available in the demo yet. The tool seems quite similar to Robillard's FEAT; the main difference is that FEAT targets Java exclusively (I think), while CME is meant to be generic.

The critical bit (and the one I wasn't able to get much detail on, though CME is open source so I could dig into it myself) is the quality and expressibility of the model and query language. The model is, of course, graph-based, but it's not clear how it handles collections, identity, etc. The query language can do basic structure and pattern matching, and should handle transitive closures in principle. It's all starting to sound a lot like my own Braque -- I wonder if I could bring my code up-to-date and compete favourably with these "new" tools. If I only had time...

I'm at ICSE in St. Louis, with a brand-new laptop courtesy of IBM Toronto Labs CAS. It's the third day of the conference (already!), and I thought I'd give live blogging a try; it should help me stay awake in some of the sessions, and provide a record of what I thought was interesting for later followup. The writing won't be very polished, but if you can get something out of it, you're welcome. Perhaps I'll catch up on yesterday's sessions and the tutorial tonight, for completeness' sake...

Object identity in sequence diagrams

When reverse-engineering UML sequence diagrams statically it's difficult to figure out which method calls are going to the same object, and which are going to different ones. Just looking at variable names is not enough due to aliasing. Using "naming analysis" we analyze the flow of references and, in many cases, can figure out that two variables actually point to the same object. The technique is fast and produces reasonably good results, though it fails on code with more complex control flows.

The author also emphasized that they're using static RE since it's not possible to get all the information out of dynamic traces. This is a good point, and means that Reef will probably need to do both static and dynamic RE for sequence diagrams. This is probably too much to fit into the dissertation... :-(

Binary refactoring

A post-source code transformation step for applications, to apply structure-changing optimizations without messing up your source. The declarative transformations must be manually defined by the developer. Probably useful in specific cases where performance is important and unachievable using standard optimization techniques, but not very exciting overall.

Automating component upgrade propagation

When libraries are upgraded, the client code needs to be updated manually to match the changes. However, if instead we capture the (interface) changes, we could automate many of the matching changes in the client. We can do this by recording refactorings -- at least the ones that were applied using the Eclipse refactoring facilities -- then replaying them at the client. Replaying is tricky, since we need to make Eclipse believe the original library source code is there for the refactoring to run. Also, this is not precise, since not all changes are formal refactorings, and not all refactorings change the public interface of the library, but should still reduce the effort needed at the client side.

The problem is that you need to capture a trace of refactorings performed, and read it on the client end, necessitating matching IDEs and plugins. What if, instead, we used original analysis to reverse engineer the most common rename/move refactorings? Not as precise, but less coupling. With a nice UI to let the developer fill in the blanks it could save almost as much effort.

The authors proposed an IDE-independent XML format for recording refactorings. This would be great, and would also benefit RE tools greatly!