Netrik Web Browser

Netrik (antrik's Internet Viewer/Browser/Explorer/Navigator/whatever) is a simplistic text mode (character cell display) web browser, similar to w3m or Lynx.

Netrik is Free Software published under the GNU General Public License (GPL), version 3.0 (or later), which essentially states that you can do anything with the programm and its source code, as long as the result passes on the same freedoms.

Netrik runs on GNU-based systems such as GNU/Linux and GNU Hurd; and with a little extra effort, probably also on other modern UNIX-like systems.

Netrik has quite limited functionality compared to some other text mode browsers, confining its usefulnes as a generic web browser -- however, it does come with some unique features too, which often make it more convenient to use in cases where it does work. (Mostly documentation in HTML format; and also some select sites on the internet, such as certain news sites for example.)

History

If you look at development activity over the past couple of years, the project will appear pretty much dead. Here I'll try to explain how this came about.

When I started using GNU/Linux at the beginning of the year 2000, I pretty soon learned about Lynx -- the oldest web browser still around; and at that time, pretty much the "standard" in text mode browsers. I originally tried it only out of curiosity -- but I soon found that I actually like browsing the web in a text terminal. (Especially on the Linux text console.) I soon started coming up with ideas for a more full-featured browser, that would allow access to a larger part of the web in text mode.

A little later I learned about w3m, which at first glance seemed to be just what I was envisioning -- though I soon discovered that it was still missing many important things. Also, there were some ideas brewing in my heard regarding innovative user interfaces, which no existing browser provided.

Yet a bit later I discovered Links -- a clone of Lynx, but with various added features (similar level to w3m). It had a less convienient user interface than w3m though; and also, at that time it looked abandoned.

So in 2001, I set out to create netrik; originally with very ambitious goals: to become the best (most full-featured) text mode browser around; but also with unique ideas regarding the user interface. In the beginnig, development progressed at a good pace, and it looked like netrik would catch up and surpass the competition in the forseeable future.

However, some 1 1/2 years later, several revalations dampened my enthusiasm. For one, w3m was gaining various new features, including some that I thought would be unique to netrik. (In spite of the author stating earlier that he didn't intend any major additions...) At about the same time, the original author of Links came forward with Links2, which has been "secretly" in development for some time -- and also came with a host of new features. Last but not least, ELinks was gaining traction as a fork of the last release of the original Links... And also getting more and more features.

Ironically, this burst of activity all around turned out to be quite temporary, and the development slowed down considerably on all these projects soon after. Still, at that time it appeared like the competition was way ahead, and keeping the pace would have implemented all the features that were meant to be unique to netrik pretty soon. So I lost motivation, moved on to other things, and netrik development was slowing down considerably. I never completely gave up on it, because I still believe in some of the user interface concepts I had planned; and even as it is, I find it more convienient than other text mode browsers. Still, other things appeared more important, and netrik development grinded to a halt.

The worst thing is that the stall came just at the time when most of the heavy ground work has been laid, and it shouldn't take too much effort to implement many of the missing bits that are limiting netrik's usability...

Unique Features

Though not nearly as much as originally planned, netrik does provide a number of things setting it apart from other text mode browsers.

Page Navigation

Netrik provides a streamlined page navigation concept, with a comprehensive number of useful keyboard commands, making navigation of pages (especially with many links) more convenient than with w3m and especially Lynx/Links/Elinks. (Most of the keyboard commands are inspired by the Vi text editor, so Vi users should feel right at home. However, a web browser is not a text editor -- and thus the way cursor/page motion is performed actually differs in quite fundamental ways. It's more like a mashup of less (the pager), Vi, w3m and Lynx; with some rather unique concepts thrown in as well...)

History Navigation

Netrik also provides some perks regarding the back/forward history functionality. First of all, it actually does offer a "forward" command -- other text mode browsers tend to omit this for some reason. While it's clearly not needed quite as often as "back", it's still useful often enough that I for my part perceive it's presence as a substantial increase in comfort...

In addition to going back/forward just a single page, netrik also has commands for going back/forward to the last time you crossed a domain name boundary. This may be somewhat arbitrary, but comes in handy surprisingly often.

In a similar vein, there are also commands for jumping all the way back to the first page in the history, or forward to the last. Again, once you get used to it, it actually turns out to be one of the most often used commands in certain situations...

Last but not least, if neither of these two last options fits the situation, you can also manually set a marker on any page, designating it as an explicit quick jumping target.

Heuristical Link Reactivation

Somewhat related is netrik's special link reactivation behaviour when going back (or forward) in the history. Netrik doesn't cache visited pages; so a page will often have changed when returning to it (especially on news sites...) -- if netrik was only naively saving the absolute page position like other browsers do, the cursor would often end up on a different link than the one that was active before leaving the page. Netrik however turns this around into an asset, by using heuristics to recognise the previously activated link even if its position on the page has changed. Thus you always get the most recent page, but still can convieniently navigate the links one by one.

Rendering

Netrik takes quite an unusual approach to rendering HTML markup: often using explicit markers to designate certain HTML elements such as forms, tables, images etc. Lynx and w3m also do this to some extent, but netrik does it much more.

In some cases, this has only been done as a temporaray crutch for dealing with elements that don't have "proper" layouting implemented yet -- such as tables. Most of the time however it was indeed a planned feature; really helping clarity in many situations I believe.

Colour Handling

Similar to w3m (and also more recent versions of Lynx), netrik uses text colours to indicate certain special elements on the page -- such as links, form elements, images, emphasis and strong emphasis etc. However, the use of colours in netrik is somewhat more extensive and more logical I believe. (Not quite as colourful as newer Lynx, but again clearer and thus more useful.)

The default colour scheme of netrik is more readable on dark backgrouds -- with another separate scheme for bright backgrounds. There is also some extra effort to make both bright and dark backgrounds work well in any terminal.

Another small but important advantage is in the way the active link is highlighted using a non-obtrusive yet well visible change in background colour.

Coloured dumps

Somewhat related, netrik has the unique ability to create a dump of a HTML page with colour escape sequences. (Using the same colour coding of special elements as when viewing in netrik.) Such a dump can be saved to a file, and later viewed (in color) by printing it directly to the terminal, or using less -R.

Anchor Highlight

When following a link to an anchor within a page (designated by a "fragment identifier" in the URL, i.e. a part of the link URL starting with '#'), netrik positions the page such as to show some context lines before the anchor (so it doesn't end up at the very top of the screen), while marking the exact location of the target anchor in a clearly visible way. I find the lack of such functionality in all other browsers I know (text mode and graphical alike) to be a major usability shortcoming.

Performance

Netrik is fairly quick in loading/rendering large pages. Last time I did a comparision, it was not quite record-breaking, but close to the fastest text mode browsers, and much faster than some others. (Also faster than graphical ones -- though that's somewhat of a given...)

In case anyone is interested, in netrik most of the time is actually spent just repeatedly reallocating storage for strings growing one character at a time... This should be quite easy to optimise, at which point netrik would indeed become the fastest browser of all, with a good margin.

Admittedly, with today's machines, I'm not sure this really makes much of a difference...

HTML error reporting

Netrik puts some emphasis on printing verbose diagnostics about HTML errors encountered.

At the time, this seemed like a good idea... Though to be honest, it hasn't turned out to be all that useful in practice; and it's probably rather pointless nowadays with HTML5 standardised error handling...

Major Shortcomings

And now for some of the things most severely limiting netrik's usefulness in the real world...

HTTPS

With more and more sites adapting a HTTPS-only policy, this has become the most common blocker for using netrik.

I'm not sure whether to try implementing HTTPS natively (with libgnutls or libnss), or better go back to the original plan and use external tools (wget or curl) to fetch pages... The latter not only have full support for HTTPS, but also for newer HTTP versions (1.1, and soon 2.0 I guess); other more obscure protocols; and probably some other things I can't think of right now. OTOH, integrating such tools seamlessly in an interactive program is quite tricky...

UTF-8

When I first wrote netrik, most sites in English just used ASCII; while most other sites used the respective national standard charsets (e.g. iso-8859-1), which usually matched the local terminal charset -- thus, we could pretty much totally ignore the charset, simply treating it as some kind of extended ASCII, and usually it would just work.

Nowadays, most sites (English and non-English alike) use UTF-8. (Even English texts now are usually sprinkled with non-ASCII characters, such as &ldquot;, &emdash;, … etc.) Most terminals are UTF-8 now as well, so this doesn't result in total garbage -- however, as non-ASCII characters are represented as multi-byte sequences in UTF-8, netrik's length calculations are off; so you get layout errors nevertheless.

UTF-8 support shouldn't actually be too hard to add; and in fact arbitrary charset support shouldn't be much harder either, as long as we accept a little performace penalty for other charsets...

Base URL

Some pages set an explicit "base URL" (different from the location where the page was loaded from), which should then be used for calculating the target URLs of non-absolute links -- otherwise, all relative links on the page are broken.

This affects quite a few sites; and it shouldn't be hard to fix.

Cookies

This used to be near the top of my list of shortcomings -- however, nowadays I'm not so sure about this anymore. The truth is that full cookie handling is quite complex, especially if we also want to implement cookie blocking (which we should); while at the same time, most of the sites that really need cookies nowadays also tend to rely on JavaScript -- and thus wouldn't be usable anyways...

Table Layout

Although the layout engine has been designed (and the fundamentals implemented) for complex two-dimensional layouting (such as tables), I never got around to add the actual table layouting...

Initially, this wasn't actually all that bad (and thus I wasn't in a rush to fix it): at the time, most sites were still (mis-)using so-called layout tables extensively for arbitrary positioning of page fragments. Not rendering these pseudo-tables indeed usually made pages *easier* to navigate...

However, actual data tables also do exist of course -- and these would really benefit from proper row/column layout. For very simple tables, the markers netrik displays (instead of aligning cells) are good enough; but for anything larger/more complex, it quickly turns unusable.

(It should be noted though that the way w3m displays borderless tables is also pretty unusable in many cases. With layout tables being mostly a thing of the past, nowadays we should probably just assume that all tables are genuine data tables, and always render them with borders...)

What about other features?

There are various other features missing, that arguably could also be implemented in a text mode browser (and were in fact part of the orginal plan), which however do not show up on my list of major shortcomings anymore for various reasons.

CSS

This is a big one -- and it was originally planned to be one of the big features for netrik. (No other text mode browser implemented CSS at that time as far as I know.)

Obviously some CSS attributes -- such as most font properties for example -- can't work on a pure text terminal. (Though sometimes it would be nice to still keep track of them internally, and show *some* kind of marker, as changing fonts sometimes do carry relevant information... But I'm not sure there is any reliable way to handle this.) Other attributes however -- like positioning of page elements for instance -- could be approximated pretty well even on a character cell display.

At first I believed that we should attempt to make pages look maximally close to the representation in a graphical browser; and thus implementing as much of CSS as possible seemed clearly desirable. However, over time I came to realise that most pages are actually easier to navigate when *not* following the layout... Further considering the *huge* cost of CSS support in terms of code complexity and performance (talking several times slower rendering and much larger memory usage here), I'm no longer sure we should strive for CSS in a text mode browser at all.

JavaScript

That's an even bigger one. And it's even more ambivalent, too... There is a lot of aspects to consider here.

On one hand, we have a strong case for implementing JavaScript, as many sites are simply not usable without it, even though they would otherwise work just fine in text mode. And it's really tempting to start adding some support, because it's often just simple things not working -- such as forms missing an explicit submit button...

However, this apparent simplicity is deceiving. While it might be easy to support each one of these straigforward cases by itself, there are *lots* of them, each a little different; and then we have the *slightly* more involved ones; and then... Where do we stop? JavaScript is *huge* (in terms of complexity/performance); and we pretty much need to implement all of it to have somewhat reliable handling of even simple things.

What's more, JavaScript has heavy dependencies in turn -- full DOM support is needed for a lot of scripts; full CSS support too (JavaScript actions are often implemented as DOM manipulation); and even then, many scripts still wouldn't be usable, because they rely on a certain graphical representation...

Another problem is that while well-designed sites should have fallbacks for no JavaScript, there are no fallbacks for partial JavaScript -- so an incomplete implementation very often can actually *hurt* the usability of sites...

All in all, my impression is that JavaScript support is pretty much all or nothing. And is implementing full JavaScript support really worth the cost? After all, the sites which do require non-trivial JavaScript are very often the same ones that are too complex to make much sense in text-only mode anyways...

Windowing

Browsing sessions tend to "branch out"; and thus a desire for some support of multi-windowing -- with things such as split view, background windows (aka tabs) etc. -- seems inevitable.

My intention always has been to implement such functionality in netrik. However, what is the *right* way of handling this? In all honesty, a browser (or any other application for that matter) has no business doing window management -- that's the job of a window manager! And in a text mode environment, this role is usually filled be the GNU screen utility.

Indeed over time I became quite used to just launching a new screen window and starting another netrik instance whenever I want to "branch out". This could be made somewhat more convenient with a builtin command that does just that. (Similar to w3m's "launch external browser" command I guess.) I don't think there is any more integration we can do with screen.

Admittedly, it surely *would* be nice to have some extra features for improved windowing system integration -- *if* we ever have a more advanced text mode windowing environment that actually allows for that... (Which is indeed one of the things I'd like to create some day -- but realistically speaking, this is not very likely ever to become more than a pipe dream :-( )

Graphical Mode

It may seem strange suddenly to see mention of a graphical mode, after all this talk of netrik as an ultimate text mode browser... Yet it really makes perfect sense: while browsing in text mode offers certain advantages, some pages just don't work very well without graphics. I often find myself using Firefox for things that for the most part would work perfectly well (and much more conveniently) in netrik or w3m; yet the cost of going back and forth whenever hitting the limitations would be just too high. Having the ability to switch between both modes seamlessly would be a major boon, actually making extensive use of text mode browsing much more tenable.

In the very beginning, I was in fact thinking that if netrik development really, *really* took off, it might ultimately become a competitive full-blown graphical browser as well. There was a bit of a vacuum in this area, as Mozilla (the first full-blown Free Software browser) was not quite there yet at that time, and it wasn't entirely clear whether it ever would be... Of course this consideration became meaningless when Mozilla was finally taking shape somewhat later.

What about these elusive user interface ideas I keep mentioning?

They are really mostly just more or less vague ideas; nothing much in the way of existing functionality I could write about. And I'm sorry to say that while I have a nice vision of various aspects in my head, I'm no good at explaining these... You will pretty much have to wait and see for yourself -- if I ever get around to actually implement them. Until then, consider it vapourware.

(Indeed I once started a blog in the hope that it would help me communicate these very ideas... However, I just ended up getting sidetracked with more and more barely related topics, and for the most part never quite got to the really relevant concepts :-( )

Status

Netrik currently is in beta for a first, quite incomplete, but kinda stable release. It's been in that state for more than 10 years now... As I said, I'm not exactly working on it actively anymore.

The last thing I worked on was a major revamp of the back/forward history code, which should result in much cleaner code, along with doing away with various bugs and unexpected behaviours in that area. I only got half way through though.

To be honest, my major demotivator for the past couple of years has been the difficulty of transitioning the repository from CVS to Git without loosing merge history...

Getting netrik

The source code repository, tarballs, and various other developer resources are all available through the netrik project pages on SourceForge.

What happened to the old site?

If you visited the netrik site before, you might have a recollection that there used to be a somewhat sophisticated setup here, with several pages and even more rambling. Pretty much all of this ancient stuff was desperately irrelevant, obsolete, and incorrect by now -- so despite my own dislike for URLs disappearing from the net, I thought it prudent in this case to get rid of the whole thing. If I ever happen to get back to active netrik development, I might resurrect select bits of the previous content in some new shape.


Download netrik