Thomas Gossmann

Unicyclist. Artist. Developer.

PDT and Composer: A visionary concept

It all started, when I sketched this UI concept for a possible eclipse composer plugin:

and also created a pull-request to start the development on the editor. For now, it looks pretty much to my first sketch. In the early days of the composer plugin Robert and me fastly realized the potential that comes with the plugin. We created issues for our ideas, unfortunetly least of them come to live. Now, there is a discussion to merge this plugin into PDT itself, which is an amazing step forward, as I think this is the right place for the composer functionality to sit at. I would like to use this to discuss the original visionary concept behind the composer plugin, which I think should be handled if a merge into PDT is happening (as it would be the right time to get hands on this).

The Benefits ahead

Despite the currently implemented features, there is so much more.

Enterprise features from JDT

Eclipse JDT has a lot of features that could be transported over to PDT, which are currently realized through PEX where the composer functionality has an indirect influence on. They scratch the idea of enterprise support and that’s what the composer plugin is all about: delivering enterprise-level functionality. These are:

  • New Class/Interface/Trait dialogs (partial available in PEX, should be built-into PDT).
  • Refactoring (Zend kindly donated their Refactoring functionalites from Zend Studio to PDT)
  • Namespaces in the Project Explorer. This is an old feature of PDT, which got paused for a long time but is revived in one of the latest releases (In the little triangle in the Project Explorer, select the „Group by Namespaces“ Option).
  • Composer Dependencies listed in the Project Explorer. It should be available next to „PHP Include Path“ and „PHP Language Library“. It could look like this:

 

  • [L] Composer Packages
    • [P] Symfiny/Finder @ 2.7
      • [N] Symfony\Component\Finder
      • [N] Symfony\Component\Finder\Adapter
      • [N] Symfony\Component\Finder\Comparator
      • [N] …

(Legend: [L] Library Icon [P] Package Icon [N] Namespace Icon)

The packages tree would list all installed packages along with their installed version number. Once you expand this, you see all the Namespaces that are available to you via this package and on expanding those you see classes, interfaces, traits, etc. This would be a very nice addition to the project explorer and solves the hassle of digging into the vendor folder, when looking for a particular class of a certain package, while simultaneously blending out all the „clutter“ files of these packages.

References on this section:

Local Development

Imagine you have two packges, package A and package B. Package B depends von package A and guess what? you are developing both of them in parallel. So the dependency is just a folder away, however in order to send changes from A to B you need to roundtrip packagist (or your own package management system). There is a delay for each change, that is very inefficient programming. I wrote about Composer Development with local Dependencies earlier this year, you can read the whole story there. Meanwhile I did some experiments over at composer-localdev-plugin to address the forementioned problems (not very successful I must say so far when it comes to details; but good progress how to address the problem in general). For this to realize there is more developm93ent on 3rd party ends necessary.

References on this section:

Diagnostics

There are a lot of ways, an IDE can help diagnose problems with your composer installation and assist with packages. The IDE can help:

  • Checking whether packages are up to date or if updates are available
  • Security Advisor check
  • Assisting with dependency-hell
  • Debug mode for the graph (partially implemented today)

Also I can see a nice Packages page for the editor, which can look like this:

References on this section:

Managing global Packages

The composer editor is used to manage packages for the current project. However, the same editor or respective parts can be used to manage global packages, that are placed in ~/.composer/composer.json. Would be interessting to see, where it would go. A view? An editor? This needs to be researched a little better, also from a usability perspective.

References on this section:

API for tooling Plugins

With the globally managed packages it would be an easy task to install php tools via composer in a global scope. Think about PHPUnit. You might have it installed via composer global require 'phpunit/phpunit'. In your package folder you just type $> phpunit and start your testing. Using the global command and local phpunit dependency (just for ease and being lazy instead of typing the longer local binary path – which works as long as phpunit is installed in the same major version). That’s how you run your tests, that’s how they run on travis or your CI of choice. That‘ how your IDE should run them. Now, think this ahead and imagine the following:

Install a PHPUnit plugin (or at best it ships with PDT itself but sits on top of the composer functionality). You can install PHPUnit globally with PDT (or it happens in the background, because it is handled for you just in case it is necessary – minimize the mental burden of the programmer, just because we can and we are nice). So the PHPunit command is run, from your IDE and displays the result similar to the JUnit testrunner of JDT while actually using your underlying architecture you would use when manually issuing the command from your shell – how awesome would that be? No phpunit version maintained by PDT or any plugin which basically means it is outdated soon. You can even go further and get line-highlight for code-coverage similarly to how EclEmma does that for Java code. Then we are touching absolutely awesomeness.

References on this section:

API to extend the Composer Editor

Specific package types may come with special tools or utilities. It would also be great to give them a place inside the Composer editor. Would be great if plugins can extend the composer editor with additional pages and may have a choice to show them only, when a specific package-type is given (they are hidden anyway).

On the other hand it would be wise to expose the package type to eclipse so plugins can react, e.g. by decorating the project explorer with framework specific icons.

References on this section:

Exceeding PDTs Limits

PDT is somehow limited in a what it knows about its project. Most basically, this is a php package and it runs on php version X.Y.. Composer provides a lot of metadata for each project, especially through the autoload configuration.

Namespace Resolution

For any given path within a project this can be resolved to a namespace (if defined in composer.json) and even vice versa. This is a great addendum to the aforementioned new class/interface/trait dialogs in combination with „group by namespaces“ option. Whichever presentation is active in the project explorer, right clicking a node and opening a new class/interface/trait dialog, just from the selection it is possible to get the path respectively the namespace to completely prefill all required fields in the respective dialog – hell yeah!

Project Layouts

PDT as of now only knows about two project layouts. First: Put everything in the root folder into the build path. Second is the creation of an application and public folder. There is not very much variation. It would be nice to have a layout for each framework, however this would end up in a mainteneance nightmare and would probably result in a customer dissatisfaction because they would be outdated very quickly. Composer can close the gap and the composer functionality can be used to bridge this into a very handy user interface. Framework authors can provide their project layout though composer and IDEs can consume them (with the following command: composer create-project vendor/layout). However, everybody invovled would need to agree on certain package type. For each „new project wizard“ a page with project layout can be displayed and programmers can search for their desired framework and are provided with the package that serves as project layout. This search would look similar to the dependency search in the composer editor however it is only search for the agreed package type.

References on this section:

  • Composer-Eclipse-Plugin: CEP#38

The Challenges

The Composer-Eclipse-Plugin depends on a lot of dependencies and merging them into the eclipse ecosystem is the most challenging thing despite refactoring the top packages to their new appropriate names (laughing). Here are some dependent things, that are made for only the sake of the Composer-Plugin but are suitable to serve at a higher level.

Eclipse Elements/Widgets

The original idea for the composer editor was the eclipse plugins editor and a lot of concepts (and code) were taken from them, especially when it comes to create the UI (in order to stick with the eclipse-theme). I identified recurring patterns and extracted them into its own little library, the eclipse-components library. Most of these concepts are used in the composer editor and I resued them (after extracting them in my eclipse-transpiler-plugin). For now, they live side-by-side in the Composer-Eclipse-Plugin and the eclipse-components, yet the latter ones are more accurate and contain more options. These are UI-Elements that occur everywhere in eclipse, I think it would be a good idea to have a place at eclipse (close to the SWT group) that would manage them and make them available to every plugin in the eclipse ecosystem. For now, the eclipse-components are still built on the eclipse 3.x API which should need to be modernized to eclipse 4.x since especially the new DI mechanism are more appropriate to this purpose, especially since all the UI elements can be turned into POPOs. I even started using the eclipse-components to refactor the composer-editor UI in a local branch, which revealed some problems (I expected the elements to work flawless – how could I?).

References on this section:

Running CLI-Tools from eclipse

When developing the composer plugin we had a little hassle of executing the composer executable and wrote some workarounds for it, which I also extracted to the eclipse-components and used it in the eclipse-transpiler-plugin. It is actually very tricky to run cli-tools from inside eclipse. The reason is, eclipse uses its own environment. It doesn’t have access to your local ~/.profile, ~/.bashrc, etc. with defined $PATH variables, etc. Most basically, cli scripts start with the following line:

#!/usr/bin/env php

Eclipse isn’t aware of this env(ironment) and as such isn’t able to execute this script. The trick is, to prepend the script with the executable of the environment the shebang mentions. Calling which php would give us the path to php binary: /usr/local/bin/php. So calling the above script (assuming its name is phormat), the call would look like this:

/usr/local/bin/php phormat

In case of PDT we have the path to the executable given and could easily use the information from the PDT plugin for this. However, this affects every cli script we use in our current toolchain today, hence think about how many tools we use that run on node, the same problem persists across all these. So it would be very keen of eclipse to provide an API to call cli tools and my first assupmtion would be to have it as part of the dynamic language tools kit (DLTK). Here is how a possible call for an execution can look like (in schematic fashion):

  1. DLTK.executeCLI(„/path/to/script“)
  2. DLTK itself reads the first line, checks if the shebang starts with #!/usr/bin/env
  3. Lookup for the appropriate binary.
  4. If no predefined binary can be found (see below), it calls itself which <binary> as a fallback solution to ask the system what it got for DLTK
  5. Finally executes the script

Extension Points for DLTK

Plugin extension points can be used to provide binaries to DLTK. E.g. PDT can register at DLTK and providing the php binary. These binaries can be managed at global preferences, where DLTK provides a nice UI to handle that.

Conclusion

This way, scripts can be executed from your favorite shell and from your IDE while almost using the same execution. We as devs want to minimize environment differences between shell and IDE when executing scripts in contrary when a IDE (or plugin) keeps a certain binary that gets outdated over time, just to provide this functionality. Also it would come in very handy for other/future plugin authors to use this architecture to build their own plugins for specific tools.

JSON-Editor

The composer plugin relies on a JSON editor which is used to display the raw composer.json. We couldn’t contact the original author of the JSON plugin and decided to republish it as part of the PEX update site and maintain some problems to make it work with the composer plugin. It is rather hard to move this part in eclipse, especially with aspects to IP and licensing. On the other hand due to its practically unmaintained state there are some bugs (e.g. the outline often reports an error) and missing features (identation and formatting options). This could be concepted from scratch and find an appropriate home within the eclipse ecosystem for it (I would suggest WTP). Since json isn’t used exclusively for the web anymore it is a very nice addition to eclipse anyway.

Composer-Java-Bindings

The composer java bindings basically provide an API to all the composer fields that can occur in a composer.json file. Despite that it also is used to format the current output of the composer.json which actually should be handled by the json editor (see above). Though, it can used to proper order fields in the json object prior to formatting (I would even like to see this). At the moment the library is availabe as a maven project which somehow (this is the magically part for me) needs to be accepted within the eclipse ecosystem. The java bindings also make use of a library called json-simple which also need to be made available. (I literally have no idea how this works and was glad Robert took over this job of managing all the dependencies needed for the composer plugin).

Looking forward

This post describes the ambitions plans for the eclipse composer plugin at enterprise level, some of the struggles that we discovered, which workarounds we used during development and what challenges lie ahead in order to merge this plugin into PDT itself.

Plans for the future are now open for discussion.