Overview
This is a proposal for a new XD plugin subsystem we’ll call Scripter (as a placeholder; another possibility is Scripsit–Latin for “I have written”), using the present tense for simplicity (none of this exists). All the details are subject to change and to further development, as this is a very early stage proposal. And, of course, all these details would take some time to develop, starting with a basic “minimum viable product,” expanding the feature set over time.
Scripter provides a panel where both end users and developers can interactively discover, run, edit, and debug automation scripts that drive XD via its various APIs. The scripting language is at heart Javascript, but in a potentially simplified and stylized version suitable for easier understanding and development, and with higher-level APIs for things like synchronous network and file access.
Scripter also provides a set of “scenarios” pre-programmed for common cases, where the user only has to fill in some blanks. E.g., scanning all scenenode objects (recursively) in the selection, and potentially modifying certain node type’s properties.
Scripter users have access to an online, constantly growing/evolving set of useful, public, curated scripts to run, modify for their own needs, and learn from, along with a robust discussion forum.
Motivation
The motivation for Scripter is fairly simple. Non-developers need access to automation that doesn’t involve full Javascript/UXP/XD API knowledge, external editors, packaging, console.log (or CDT) debugging, etc., and with a higher-level API for things like file and network access. And they need hand-holding, so Scripter would provide specific high-level help at most points as to what’s needed or what’s legal.
Also, if you look at the range of plugins now available (July 2019) for XD, easily 80-90% provide almost trivial functionality beyond the built-in UI, and nearly all those could easily and better be provided by scripts.
Scripter’s existence promotes a huge, flourishing, curated ecosystem of scripts for a whole range of automation and going-beyond-the-built-in-UI needs. And this is orders of magnitude beyond what plugins could provide, since there’s a natural barrier there for non-developers. (As low as it is for developers, it’s still mighty high for graphic designers.)
Plus, it makes a lot more sense (in terms of discoverability, etc.) to have plugins be the venue for substantial products (in the hundreds) rather than the (tens of?) thousands of “one-off” kinds of little widgets which are better implemented as scripts, and whose presence is already greatly muddying the plugin channels.
Scripts
Scripter’s ecosystem of crowdsourced scripts would be initially primed with a whole set of useful example scripts, and then grown over time (with human curation) by direct submission of scripts from the Scripter panel, and submission of modifications to others’ scripts. (Perhaps using Github for forking/submitting, etc.)
Scripts can also be privately shared among a given group of people (e.g., a project or company), and simply stored locally (even if downloaded) for when sharing doesn’t make sense.
Script files (.scripterxd extension, for example) are textual (though not necessarily directly editable), and can be passed around like normal non-binary files (somewhat resistant to white space changes), diff’d, stored on Github, etc.
High-level point-and-click coding
Scripter provides a Scratch-like (https://scratch.mit.edu/) coding environment using point-and-click (drag and drop?) building blocks for language elements, with only the appropriate/legal blocks available at any point. And Scripter can help with XD-specific knowledge that you have to set the top-level property of an object (e.g., a fill color) rather than a sub-member.
Of course, a script can be viewed/copied (to the pasteboard) as Javascript at any point, and experts can type to create scripts.
There’s some chance this is too ambitious, and we’d just fall back to editing Javascript code directly, though with ongoing more-than-tooltip context-aware prompting. Or we might come up with some amalgam.
Debugging
Scripter provides a full, interactive, step-by-step debugger, with introspection into the full state of the script. Updates to the scenegraph can be seen at each step, naturally.
Scripter has a “slow-mo” feature where it continuously single-steps at a user-chosen pace, to understand how a given script works.
Script safety
Since a runaway XD plugin can hang the whole application, Scripter watches loop counts and recursion depths, and will interrupt after “reasonable” limits are reached (which can be controlled). The user can choose to continue further (up to some absolute limit).
Version control
Since scripts are generally small, Scripter maintains a complete version history (up to some user-controllable limit), so the user can quickly switch between versions with forward/backward history buttons to visualize changes.
Optionally, Scripter will connect to a user’s Github/Gitlab account, and store its scripts there for complete version control using standard tools.
“Watch me” feature
Scripter has a “watch me” feature, where each change made in the UI to a given scenegraph object is reflected in code created by Scripter to match the UI change (relative or absolute for numeric quantities), for easy coding of property changes. (Within some limits, obviously.)
“Live” coding
Scripter can also operate in “live” mode (with some obvious limits), where you’re working with (and inspecting) real data as you code, and seeing the effect on your data and on the scenegraph, at each coding step. There’s nothing more powerful than working live, when possible.
In live mode, Scripter will work with partial programs, and simply stop and let you fill in unfinished portions, based on the live data at that point.
Easy data-driven document building
Scripter makes it easy to build custom data-driven applications with built-in, high-level access to things like databases, Google Docs sheets and docs, full JSON and perhaps GraphQL support, etc.
Simple UI building
Scripter comes with a set of prebuilt UI objects (e.g., text input, radio buttons, value sliders) that act just like normal building blocks but provide input from the user when a script is run.
Beyond that, Scripter provides a set of real dialog building blocks that could be assembled to cover many basic UI cases.
Scripter’s goals do not include handling all possible UI scenarios; at some sophistication level, a plugin would be more appropriate.
For plugin developers as well
Even seasoned plugin developers would greatly benefit from development exploration in Scripter, with live data and context-sensitive help.
Then, Scripter can ‘eject’ a script into a fully-working plugin at any point, for further, more sophisticated development.
Cross-product
The core of Scripter is designed to be cross-product, with versions for XD, PS, IL, ID, etc., each version customized for the point product’s API, as those products come online with UXP over the next few years.
Scripter for each of these point products accesses a different universe of public scripts and discussions, since there isn’t much (if any) cross-product scripting given the differences inevitable in the UXP-based APIs.
(There’s probably some sharing possible for network/file high-level utilities.)
Collaboration
I (we) would be delighted to collaborate at every step with Adobe personnel to help shape the product development in a way that maximizes the benefit to Adobe’s end users and developers.
I don’t see any big obvious technical roadblocks to developing Scripter for XD using the existing APIs, though I’m sure issues will come up.
And I’m sure we’d need occasional fairly high-level support from Adobe for gnarly problems, up to and including APIs fixes and changes to maximize Scripter’s potential. But that should fall out naturally from the collaboration envisioned.
A simple example of potential collaboration: Scripter’s “watch me” mode would tremendously benefit from a new panel entry point, “command invoked”. This callback would just let a panel plugin observe every XD UI-level menu command invoked by the user. And it would be useful in general for plugins to know what’s going on at a global level, and not just by guessing at what happened by inspecting scenegraph changes and hoping for the best.
If we pull off even half of these ambitious goals, Scripter will easily be the among most sophisticated scripting environments ever developed. And that can only mean good things for XD and Adobe.
How to support?
The obvious desire is to have the Scripter plugin generally free to use, to encourage widespread use and general applicability.
The big question for Em is: How to support the development of such a product, which is clearly some man-years of work in the long term?
One approach might be having Adobe underwrite the effort for for the first 2-3 years, coupled with selling “Pro” licenses that incorporate the higher-end features.
Long-term, once kick-started, Em could sell low-cost quarterly/annual subscriptions to the crowdsourced scripts/discussion back-end, with some limited access at no charge. (I.e., use the WordPress/Automattic model.)
This isn’t the way to fast riches, but we’re interested in a long term solution that benefits everyone. Such a solution also needs to produce enough income that Em can continue to develop and improve the ecosystem for years to come, and not just end up with a “one-off” product.
Why Em?
My (Chris Ryland’s) academic background (Harvard/Columbia CS) is in languages, compilers, and interpreters, with a decade of networks & operating system development experience and leadership after school, and now almost 30 years of deep electronic publishing software industry experience founding and running Em Software (with 5 years in the nascent laser printer industry before that).
I’ve done a lot of thinking/scheming over the past few years about similar systems for InDesign, though more with an eye to data-driven publishing, and less general. And I’ve also designed a whole more general dataflow system called VisiFlow, which shares some properties with the Scripter idea, but which is likely overly ambitious to be tackled with Em’s limited resources. Scripter seems more in line with what’s actually possible, but general enough to be challenging and interesting.
Since the beginning of 2019, I’ve done a deep dive into XD panel development, and have created three non-trivial plugins. One of them (Layers for Developers) required exploring every nook and cranny of the XD scenegraph API. So I’m now completely conversant with UXP and XD API usage. And with decades of deep experience with many different languages, from assembly to SmallTalk (most recently Javascript in depth), I’m confident we have the expertise to build what we’re proposing here.
In particular, building and working on dynamic language interpreters is something I’ve done before, so that’s no real challenge, even if rather large in scope.
Behind me are two very senior software developers/architects (Chris Roueche, John Whitney) who could be called in to help with various aspects of the Scripter product, as it starts to mature.
We’d also expect to add personnel to the project (slowly), if the Scripter project proves its worth long-term.
