Custom Build Plugins / Layer and Documentation ideas

Submitted by humbletim on Thu, 07/31/2008 - 02:56.

I’ve been pondering a few ideas relating to the build and jsdoc systems. I’m just sharing the early concepts here for open discussion, if anyone else is interested. Thanks to James Burke for providing insights on the current build system and pointing me in the right direction.

The Ideas:

__doc__ Python-style runtime documentation

For those not familiar with python, running it without any parameters starts an interactive shell (Firebug is very similar). Python documentation is often accessible as a string variable – for example:

>>> object.__doc__
'The most base type'

I’m thinking for jsdocs this could involve an “inverse optimizer” of sorts, which instead of stripping comments would “intern” them as the inline __doc__ variables– perhaps similar to templateString interning
thinking about two possible distribution formats:

Baked right into layer.js
A separate, aggregate layerdocs.js containing just the .__doc__= mixins

Further research still needed to identify similarities and differences between build system hooks, documentation system hooks, and dojox.help.

Automagic layer generation

Example #1

load a complex page with many dojo.requires, then run some dojox command to generate a fully functional profile.js (based on the actual page’s module usage -- including some comments for acquiring and using with build scripts)
Maybe even use cookies to snapshot modules across application pages and then generate an aggregated profile.js for everything at once

Example #2: automatic server-side HTTP consolidation

extend on common CSS / standalone .js “just-in-time aggregation” concepts – could be as straightforward as scanning content or parsing apache log files and associating dojo .js modules with pages. If fully automated (and fast), there should be no reason not to produce personalized layers for every single container page! ;)

Layers fail gracefully when incomplete (basically a noop on missing components?) – these “cache misses” can be used as a feedback loop for ongoing optimization
The general idea here is to not have to think about layers too much just to get started and access their benefits (not that it's very difficult right now, but I think it can be made easier).

Current Systems Summary:

Build System:

Started as a mix of ant, python, and javascript on custom Rhino
As of 0.9 it’s all pure javascript on custom Rhino

Doc System:

[todo – drupal / php?]

dojox.help:

[todo] xhr-backed, limited to core dojo API(?)

Login or register to post comments
Unsubscribe post

Here's a quick Perl hack to

Submitted by humbletim on Tue, 08/05/2008 - 21:49.

Here's a quick Perl hack to test out usefulness of using .__doc__ variables. It catches many (most?) function cases -- could refine this into a Rhino .js. Not very useful yet, but here it is. Using Perl to create Python-style documentation for JavaScript code -- lovely! ;)

Results on a stock dojo-release-1.1.1 dojo.js.uncompressed.js here: http://yadtd.info/dojo-release-1.1.1/dojo/dojo.js.uncompressed.js.__doc_... (gzipped)

#!/usr/bin/perl -w

#
#usage:  perl jsdoc2__doc__.pl < dojo.js.uncompressed.js > dojo.js.uncompressed.js.__doc__.js
#then from Firebug: dojo.query.__doc__ etc. to view comments

use strict qw{vars subs};

$/ = undef; #slurp

my $js = ;

sub jsesc {
  my $nine = $_[0];
  $nine =~ s/(['\\])/\\\1/g;
  $nine =~ s/\n/\\n/g;
  $nine =~ s/\t/\\t/g;
  return $nine;
}

$js =~ s,(\n([\t ]*)(var\s)?([a-zA-Z0-9_\.]+)(\s+=\s+)(function\s*\()([^)]*)(\)\s*\{\s*)((?:\s*//[^\n]+\n)+).*?\n\2\})([ \t]*[}\n]),

  my $nine = $9;
  my $str = "$1;try{$2$4.__doc__$5'";
  $str.jsesc($nine)."'}catch(e){console.error(e);};$10";

,smgex;

print $js;

As a prerequisite for

Submitted by jburke on Sun, 08/03/2008 - 03:41.

As a prerequisite for supporting a _doc_ property on elements, it seems to require the ability to parse Dojo's doc format. It might not be too bad if you are just want to get the summary, description and argument descriptions that are in the first comment section inside a function. But if you wan the ability to link to object definitions for the argument types (like how the dojo.xhr* calls uses a custom-typed args object), that will require more or the Dojo doc parser work.

So for me, converting the doc system to be javascript instead of the current php/Drupal combo seems like a useful thing to do. That will make it very easy for anyone to generate the docs. Then, it seems like using the dojox.help approach to fetch the docs might get you want you want, since the doc system support a json API to fetch the docs. The docs could live in a separate directory, so no need to include the _doc_ properties in the source itself.

javascript doc parsing

Submitted by humbletim on Tue, 08/05/2008 - 16:42.

I started playing with accessing the docs from javascript, by extending on an earlier JS pre-processor concept. Some of the emergent results here are fantastic!!

I'm now rounding up the .__doc__ assignments into a separate list (which can be more or less directly eval'd to attach the __doc__ properties).

These Firebugs were run against a simple dijit.form.ValidationTextBox widget:

{{{

>>> dijit.byId('MyVTB_0').buildRendering.__doc__
"dijit._Templated:buildRendering() // summary: // Construct the UI for this widget from a template, setting this.domNode. // Lookup cached version of template, and download to cache if it // isn't there already. Returns either a DomNode or a string, depending on // whether or not the template contains ${foo} replacement parameters. "
>>> dijit.byId('MyVTB_0').onBlur.__doc__
"dijit._Widget:onBlur() // summary: // stub function. Override or connect to this method to receive // notifications for when the widget moves out of focus. "
>>> dijit.byId('MyVTB_0').isValid.__doc__
"dijit.form.ValidationTextBox:isValid(/*Boolean*/ isFocused) // summary: Need to over-ride with your own validation code in subclasses "

}}}

NOTE: forums "Format as:" dropdown is blank so the above code may be mangled.

This solves my immediate need for runtime development triaging, but imagine now if the Method Resolution Order (Python MRO / this.inherited) were also easily accessible. :)

Need some info.

Submitted by ttrenka on Tue, 08/05/2008 - 17:14.

"NOTE: forums "Format as:" dropdown is blank so the above code may be mangled."

What kind of user are you on this site? I have a feeling that it's blank because of user privs, but I don't know what privs are causing that--and Alex didn't know either, when he ported this comment form code over from api.dojotoolkit.org ...

Search