Making embedding more useful

One of the trickiest problems electronic engineers have is with keeping their symbol libraries up-to-date while not breaking old schematics (e.g. with changing pin positions).

Currently, most people use one of two approaches to managing the problem:

1. gEDA has the ability to embed symbol data in the schematic file, making the schematic independent of changing symbol libraries, and some people choose to embed all their symbols. However, a full copy of the symbol is embedded for each time it's instantiated in the schematic, making the file size bloat enormously. In addition, not only is neither possible to check whether any of the symbols have updated versions available in the library nor to update all symbols of a given type together, it is also impossible to make sure that all symbols of the same type are embedded from the same version of that symbol.
2. gEDA also has the ability to use a project-specific configuration file which lives in the same directory as the schematics. This can be used to implement a project-specific component library where all symbols used by the project live. Although this solves the problem of keeping all symbols in the project in sync with each other, it is quite high maintenance to keep the project library in sync with a "master" library. It is also still impossible to automatically check for newer versions and update them.

My solution to this would be to add the missing features to the embedding functionality (presupposing that these embedding bugs [1692626] are also fixed).

By default, each symbol used would automatically have a single copy embedded in the schematic, which instances of that symbol would reference. This would mean that all instances of a given symbol would be upgraded in lockstep, eliminating inconsistencies.

A hash algorithm would be used to compare the embedded version of a symbol with the latest library version. If they differ, an icon would be shown in a new component manager (derived from the current component selector) and users would be able to update symbols from there, either all symbols at once or one symbol at a time.

I have yet to work out whether it should be possible to edit symbols which do not exist in the local library, but it should certainly be possible to open them for read-only viewing or export them to discrete symbol files.

I believe this solution would eliminate most of the current deficiencies of gEDA and gschem for keeping archived schematics safe and keeping current schematics in sync with libraries.

Terminal emulators, utmp and setgid

Something that's been really irritating me recently on major Linux distributions is the increasing proportion of user software being installed setgid.

This is usually for security reasons; for instance, to either protect system services from accidental user interference, or to protect the user's personal information from being sniffed by other software they may be running.

Unfortunately, when a setgid executable is run, the environment is "cleaned"; potentially dangerous environment variables like TMPDIR and LD_LIBRARY_PATH get unset (this stops library preload attacks, for example).

But what if you want to have LD_LIBRARY_PATH set, for legitimate reasons?

The first set of breakage that bugged me was in xinit, where the call to start the window manager (which occurs after the user's login scripts are run) was wrapped in a call to SSH_AGENT. This effectively meant that it was not possible to set LD_LIBRARY_PATH for your window manager session at all. See Red Hat Bug #164869. Fortunately (a) there's a workaround available (see the bug discussion), and (b) it looks like this is going to be fixed soon(ish).

The second set of breakage was that several terminal emulators (xterm, konsole) are being installed setgid utempter. /var/run/utmp is a file which stores information on who is currently using the system. In order to protect it from malicious (or ignorant) users, it's protected by being only accessible to the utempter group, so in order to be able to be able to update it, terminal emulators need to be installed setgid.

I think this is stupid. It violates the rule of least surprise, and makes it difficult to debug environment variable problems in one's window session. Most of the workarounds are ugly hacks (like setting LD_LIBRARY_PATH in .bashrc, which is not what .bashrc is for).

Fortunately, I'm not the only one who thinks it's stupid, and now a change request has been filed against utempter, although goodness knows when it'll actually get fixed. See Red Hat Bug #246063.

The workaround for these problems I'm using at the moment is environment modules. I described the procedure to set it up in a posting to the gEDA user list.

Hacking in the unstable branch

For most of June, gEDA has been in the process of switching from using CVS to git for version control, and quite a lot of developers' time seems to have been spent on getting used to the new system. I really like the new system; it makes reviewing changes and maintaining patchsets so much less of a chore, and my productivity has gone through the roof!

I've made really good progress on the component library and Scheme work which I said that I had planned to do during June:

1. The component selector preview widget now works properly for all possible component library backends. My changes coincidentally fix a security hole exploitable by malicious library distributors (they could use the library directory RC file mechanism to execute arbitrary code on your system).
2. system-gafrc is now installed by libgeda, and loads separate Scheme files for the default libraries and default font. Hopefully, this should make it easier to avoid using the default symbol library at all if, for some reason, you don't want it.
3. I zapped a bunch of deprecated Guile functions, and changed the configure scripts to check for Guile by version, but it's pretty boring stuff and I don't really have any motivation to finish it of at the moment.
4. Replacing the directory and command component library backends with equivalent Scheme implementations has had to be suspended, due to the fact that the Guile functions for working with pipes really don't work very well on Windows, and Windows support looks like it's going to become more important over the next few months.
5. I've replaced the s_clib_glob() with a similar function, s_clib_search(), which can operate either in exact matching mode or glob matching mode. This will allow people to use symbols with glob special characters without strange breakage. I've also added a caching mechanism for the results of the search. I went through all the places where pointers to CLibSource or CLibSymbol structures might be cached, and tried to make sure that they weren't; this was to make it possible to add, remove and refresh component sources without causing breakage.

I have introduced a patch to my personal tree which adds a list of currently used symbols to the component selector. I find this really useful already for speeding up adding frequently-used parts to my schematics, such as resistors and capacitors. I'm also hoping to add a button to the component selector dialog which allows you to directly open a selected symbol for editing.

My recent work (and discussions on the mailing lists) have inspired two more ideas, about which I'll go into more detail on a future occasion:

• Per-directory configuration contexts.
• Overhaul of the embedding system to make it more useful.

gEDA is moving to using git

A few days ago in response to other developer's comments/complaints on moving gEDA to using Guile 1.8, I suggested forking a stable branch of gEDA which would use Guile 1.6 while keeping the main development tree using Guile 1.8.

Ales decided this was a good idea, saying:

Here's the plan going forward:

• I do want a stable/unstable branch/release arrangement going forward.
• I do not want to use CVS to maintain this though.
• So, I am going to go ahead and setup a git repository as the official repository of gEDA/gaf.

I think this is really good news, as git is so much better than CVS in many ways, most importantly (in my opinion) the speed and ease of branching, merging and maintaining patch sets. The web interface is better too.

One unresolved problem, though, is that Ales has not yet decided what to do about committing to the new unstable git branch (he's decided to maintain the stable branch himself, accepting patches by e-mail & SF tracker). There are a number of options.

The first option is the Linux kernel model: each person has their own repository, and the repository belonging to the head honcho (in gEDA's case Ales) is the "officia" repository. In this case, the approach would be as follows:

1. A select (probably less than five) group of people (let's call them maintainers) have their own repos on git.gpleda.org (or elsewhere), from which Ales "pulls" changes. These would likely be people Ales trusts not to break anything or commit anything he wouldn't like to see in the master tree quite yet. Each maintainer would probably have a "for-ales" branch, to make sure that only changes that are ready go upstream.
2. Developers who are not maintainers and don't mind using git could use git-mail (and associated tools, such as StGit's mailing tool) to send patches by e-mail to a maintainer, who would integrate them into their personal repository. This is good for people who only make changes infrequently, or new developers "on probation". Git provides very good tools for importing changes from mbox files, so it's not too much work for the maintainers.
3. Those developers who don't like git could use git-cvsserver to browse the history and generate patches (which are again sent to maintainers).

The second option is a "shared central repository", as used by the X.org project. This is very similar in workflow to CVS, where there is a single central repository.

1. Some people would have the ability to "push" to the central repository. They could do this either using git or the git-cvsserver. People could have their own branches in the central repository, probably named "userid-topic".
2. Other developers would need to send their patches by e-mail or using the SF.net trackers.

Because the rate at which gEDA is developed is currently quite slow, I think the second option would be best for now, as it would make for a smoother transition from CVS to git. Since it is easy to switch from one approach to the other (and back again, if need be), it will be possible to experiment with other options if problems crop up.

Abject failure to install Fedora 7

I was very excited about the release of Fedora 7 yesterday, as I've been a loyal Fedora user since Fedora Core 3 was released back in 2004.

You'll notice my use of the past tense. This is because I spent most of today trying to get Fedora 7 to install. I'm extremely glad I took a full backup of my files, as my various attempts to get the installer to run without crashing hosed my existing Fedora Core 6 system and bootloader. It was extremely random; despite having verified checksums of all the installation media, run the CD checks recommended and all previous versions of Fedora Core having installed very smoothly, Anaconda would randomly die or hang at some point during the installation process. When it crashed just after formatting my root partition, I gave up.

I was saved by two things: my careful backing up of /home before the attempted upgrade, and the ability to use my other box to burn a Kubuntu CD. I now have a system which behaves almost exactly like my previous system (hey, it runs KDE and Emacs, what more do I need) and gEDA even compiles.

I'll see how I get on with Kubuntu: other gEDA developers use Ubuntu with success, so hopefully I won't find any need to reinstall Fedora Core 6.

Update: I'm now using Fedora 7, having not really "got" the whole Ubuntu/Debian thing. Installing from a USB stick got around the CD problems I was having quite successfully.