Release 0.4 for Thimble Mozilla

This is the last blog post of the “Contributing to Thimble Mozilla” series. Watch me embark on the epic quest for the “Solution” while continue bugging David Humphrey  for reviews of my code. I will also attempt to link my narration with some useful tips on how to start Open Sourcing (from my personal experience, so don’t expect them to be complete).

In my previous post, I fought the problem of incompatible styling of my code and the upstream. It quickly escalated into a huge PR with around 140 comments. Eventually, I resorted to defining custom linting rules for my file. Obviously, the rules didn’t make it into the resulting code because defining multiple sets of rules for different files is plain bad idea.

Having said all of the above, it got me thinking of how to robustly validate all of the incoming code, so contributors and reviewers could focus on what really matters – writing great code. My initial though was to put all my local rules into the .eslintrc.json file. This will allow any code to be validated before it is committed. Now, I understand that it was a silly thing to do, expecting a huge project like Brackets to accept my humble rules. So I expressed my concern to David Humphrey and he pointed me to the issue where he had already started the same conversation (probably after reviewing my previous PR). There, a lot of big-shot developers were arguing about how to solve the problem. I tried expressing my humble opinion on the topic and gently proposing myself as the person to go for implementation. Obviously, nobody payed any attention to me, thought it was expected. My real goal here was to merely make them aware of my presence, so they would eventually go to me for the implementation.

The discussion seized after consolidating on using Prettier for code formatting. A week of nagging David about this problem (and my previous comment in the issue, hopefully) led to me being assigned to this issue.

To anyone who is staring their contribution path to Open Source, I would say: “Don’t be afraid to be a little too persistent and ask express your thoughts in places where you are obviously over your head. It is the only way people will acknowledge you.” Make sure that you are saying something sensible, though.=)

I like to think that starting my code from a wrong perspective is just my style of programming. This case is no exception. I started by watching the video on Prettier suggested in the issue’s comments. It wasn’t really helpful, so I turned to my good old buddy Google for suggestions. He was really helpful in providing me with the full documentation, not really what I was looking for, but at least I gained some solid ground. Next, I had to fight a wild boar – the Grunt. It was another technology that I had no idea of how to use, so it took some more time exploring its API and different extensions. Eventually, I landed on the grunt-exec, which is command line extension for grunt. Lucky for me, Brackets code was already using grunt-exec; unlucky, it was my first step in the wrong direction.

I blissfully started tweaking Gruntfile.js to call Prettier through grunt-exec in one of the tasks. As usual, I would be glad to show you the snippet, but I have dropped the commit…

Here is my second advice: “Never drop your commits. If anything, just squash them into one and leave it there.”

In any case, my attempt was only a partial success, but partial is not good enough. On one hand, I did manage to run Prettier from console, on the other, there is a list of directories that I needed to avoid formatting; that list is defined as follows:

gruntfile_meta

The exclamation sign before the pathname indicates the directories to be avoided. The problem is, I could not make it work with bash globbing. A couple of days went by and I referred to David for advice.

My third suggestion to you: “Never be shy to ask questions. Most of the time, other people in Open Source are delighted to answer your questions. However, before asking anything, be ready to answer incoming questions about the nature of your problem”.

Back to me. After consulting with David, I was pointed in a new direction. It turns out, that there are Prettier extension for Eslint: eslint-plugin-prettier and eslint-config-prettier.

At this point I should have know that there is some sort of plugin or extension for everything.

Both of the extensions require a specific version of Eslint installed. The problem is, Brackets does not directly use Eslint, instead it relies on grunt-eslint to get the job done. So I needed to update grunt-eslint to the latest version to make it compatible with eslint plugins.

The last challenge was to decide when exactly to format the code. For now, we’ve settled on updating on every commit. This was achieved with the help of grunt-githooks. Its intent is to create a bash script that would be executed before every commit. In my case, it will runs a grunt task.

githooks_snippet_2

And here is the task itself (The full file could be viewed here):

grunt_snippet

This is it! Now every commit is preceded with the grunt task that formats all of the code and adds it to the current commit.

As you can see, it was a long and winding road that led me to the solution, so my last advice here is : “Don’t be afraid to get into a tight corner, this experience will enable to identify the correct solution.”

Here is my pull request.

Release 0.3 for Thimble Mozilla

Part 1:

In previous blog post I’ve continued talking about the challenges associated with contributions to a huge Open Source project like Thimble. During this release, I continued working on the same piece of code.

After an extensive review and a huge amount of feedback, my initial code was completely discarded and a totally different road was taken. My first implementation was focusing on adding functionality to the existing code by creating a new module that will be called together with every filesystem event. There were a number of problems with that approach:

  1. The code required a whole bunch of snippets to be added in different parts of the existing code. This would have made if a real ordeal of maintaining the new code.
  2. The resulting code was now really extensible.
  3. The solution was kind of brute force, which conflicted with the coding philosophy of the project.

So with the careful guidance of David Humphrey, we were able to narrow it down to the desired procedure. Instead of making my module a colleague of the bramble instance, we make it a decorator of the filesystem, represented by the Filer object. Here is a simple diagram that show an example with the “writeFile” function:

object_composition

  • fs_ – a filesystem instance that bramble communicates with that is actually a ProjectStats object.
  • ProjectStats – a decorator that wraps around the Filer object. It does additional bookkeeping of current project size, and forwards the request to the Filer.
  • Filer – the actual object that controls the filesystem.

With this design in mind, we are able to solve all of the previous problems. Code insertion is now kept to minimum, just enough to accommodate the decorator. Here is an example.

The ProjectStats now contains only to state variables:

  • Reference to the Filer
  • Cache that consists of key-value pair of all pathnames of all project’s files and their size.

On every call of the function that is supposed to modify the project tree, cached value for the corresponding file is created, updated or deleted. Later, when the we need to know the total project size, cached is reviewed and all of the file sizes are added.

Although, the explained solution is easily written, the styling aspect of the code posed the biggest challenge by far. If you take a look at my pull request, you will find almost 140 comments. The page even takes a considerable amount of time to load because of them.=) Most of the comments are requests to fix styling of my code. I can’t even remember how many times I needed to “fix indentation”, “add space in between the function parameters” or “delete this empty line”. As annoying it was for me to update all of that, I can only imagine how tired my reviewers were. Mainly David Humphrey. Quote by David:

@gideonthomas if you want to review this, please do so soon. I want to finish this.

 

I mean, 130 commits down the road, he was pretty pissed off and tired… So to ease his pain, I introduced a bunch of styling rules using Eslint based on feedback I received. I would have shown you a snippet, if I didn’t squash all of my old commits into a single result. You will need to believe me when I say that there were around 25 rules.

After all of this, my PR finally landed, but that was not the end of it. I needed this code for my other fix on the Thimble side. So I tried using it, and it worked fine until I logged in to the app. Once logged, the whole file tree crashed and stopped responding. I will admit it, at the time I panicked. Imagine me, a newbie developer that sneaked his code malfunctioning code into a major system. I was terrified! So I asked David to revert my PR until I can investigate the cause. An evening of futile attempts only made me feel worse. On the next morning, I was able to pin-point the problem. MY VW WAS OUT OF DATE WITH THE CODE IN MASTER!!! When I came across this problem, I was to concerned, that checking the integrity of the master branch sloped out of my mind. In the end, my code was fine (Weeh!).

Part 2:

Now to talk about a less stressful part of this release. My second problem focused on notifying users about absence of index.html file in their project root.  More details. Let me tell you this, returning to a simple html problem like that was like a breath of a fresh air. Two fast days of development ended with the following outcome:

83315dd4-251a-11e7-8f4b-62da8b07e410

This message would appear every time there is no index.html file in the root of the project, preventing consequent publishing from occurring. I am using code from Part 1 of this release to check for index file and display the message’s div accordingly.

no_index_check

So far my reviewers are content with the changes I have made. Hopefully, I will be able to land this PR soon.

 

Part 1 PR: pull request

Part 2 PR: pull request

 

Deploying your app on Heroku

In this blog post I will be expanding my previous experience of developing a web app.

Now that all of the unit testing is done, it is finally time to go online! This is probably the most existing part of the web, getting other people to try your work. Moreover, since my project is completely Open Source, someone might find it interesting and start contributing to my code with ease (thank you Travis).

First things first, I want my service to be running all the time, so using an Apache sever on my box will be unwise. So instead, I am going to use one of the hosting services – Heroku. There are a lot of other options out there like Amazon Web Services, Firebase etc., some of them might be way better than my choice, however it is free be default, it has a nice console API integrated with Git, and it was required for me to use.

After registering with Heroku, I needed to push my server code to their repo. By the way, I am running node based sever called Express. It is really fast and simple, which makes it ideal for one page apps like mine. So going back deployment, after installing Heroku console tool and logging in with my account, I needed to ask Heroku for a spot, where my code will be running, that could be done with a simple

heroku create

command. It generates a random name for app and adds it to the list of my remotes. The only thing it left for me to do is just push to that remote and Voila!

Finally, I could start my server with the

heroku ps:scale web=1

Here is the link to my final implementation: https://aqueous-wave-68137.herokuapp.com/healthcheck

To conclude, I expected app deployment to be some sort of black magic, where you needed to connect a bunch of sub-services together, but it turned out pretty well.

Introducing unit testing to your project

At some point all of us have been programming in Java, admit it, there is nothing wrong with it as long as you stopped. It was while working with JUnit testing framework, that I have first coded unit testing myself. My experience with it: “I never want to write another test in my life”. However, Java is pretty old, as is JUnit testing, and the progress is not constant.

Thanks to one of my courses I am into JavaScript now. Recently, I needed to write some testing code once more, and while expecting another grueling experience, I found myself actually enjoying the process. In spite of have “Java” as a part of its name, JavaScript bares no resemblance with it in terms of testing framework’s quality and variety.

My framework of choice was Jest – a code testing platform developed and maintained by Facebook. The main slogan of Jest is “Zero configuration testing platform”, and they live up to it. After going through the installation procedure, doing npm install and configuring package.json file, I got right into writing.

The code itself is really easy to read and pretty self-explanatory. Here is a snippet:

function sum(a, b) {
  return a + b;
}
module.exports = sum;
const sum = require('./sum');

test('adds 1 + 2 to equal 3', () => {
  expect(sum(1, 2)).toBe(3);
});

The test function is the one who runs the actual test by receiving the description of the test and the callback of the test case. Inside, we have expect that runs multitude of functions. The result of the testing could be viewed on the console formatted and easy to read:

PASS  ./sum.test.js
✓ adds 1 + 2 to equal 3 (5ms)

It just works!

I ended up implementing around 12 test functions before refraining, after all I was not doing test-driven development…. For now anyways.

So that was testing, however the more challenging part was configuring my ESlint to let Jest do its magic. You see, as a part of my previous assignment, I had to configure linting with the set of rules defined by Airbnb. Since, I am using a framework, the function that are being used are not defined in my code. So “no-undefined” rule had to go. It doesn’t sound like a big deal, and it is not if you know what you are doing, which I didn’t. I’ll spare you the details by saying that an hour of going through ESlint documentation (it is awesome) and fiddling with VSCode editor configurations did the trick.

To conclude, this week was not so much about learning unit testing, but getting to know your development environment. And it a good thing too, because now I can ease pain of my PR’s reviewers.=)

Release 0.2 for Thimble Mozilla

For this release I have challenged myself to work with a back-end of the Thimble app. The issue talks about introducing project limiting feature to Thimble.

Background:
Thimble hosts all user projects on their servers. Since, the editor is designed as a learning tool for people who are trying to get into web programming, not all of the users understand implications of information transfer between client and sever. This leads to giant projects being created be users. Every time the project is loaded, it takes too much time to transmit all of the information, so browser times out. Not a really good user experience.

The solution is still in discussion, however some common ground has been achieved. The answer seems to be an introduction of stage-based system for tracking project size, this includes possible number of files and maximum size of a single file. There should be two stages of warnings:

  • Stage one (soft warning) – once soft cap of project size is reached, a pop-up is supposed to appear to let user know that they are coming close to the maximum capacity of a single project.
  • Stage two(error) – once hard cap of project size is exceeded, an error should come up, explaining the problem and the ways for solving it (e.g. using external hosting services).

Implementation:
Once again,  the feature is not necessarily ready for the production version, so for now it will only be used for debugging purposes.

In my previous post about Release 0.2, I assumed the knowledge of my previous fix would help me here, too. I’ve never been so wrong in my life. Current task involved file handling and creation of a brand new module for Bramble. Both things I had no idea how to do. Likely, I had guidance from David Humphrey on where to start looking.

As it turns out, all of the file modification requests have to come though Bramble’s core in main.js. Some code tinkering enabled me to pinpoint exacts spots of file interactions. Bramble uses Filer project for managing its file structure. Here is a snippet:

filer_snipet

The writeFile function modifies file with a new content. My job was to record the file size before and after the update and figure out if file was appended, deleted from or erased completely. After some thought and advice from David Humphrey, I decided to settle with checkLimits function that will record the changes of the project and return the new size of the project in a callback. Moreover, I noticed that genericFileEventFn function is called every time file is being modified, whether it is being updated or deleted, so the logical conclusion would be to put my checkLimits function call there. Here this is how it looks:

projectLimiter.checkLimits_call

Finally, I just needed to figure out how to calculate file size change. Given and old size and pathname to the file, it is not really difficult. On the other hand, things are not as smooth when deleting whole file. Since my module works with the file tree post update, deleted file no longer exists. Given the circumstances, I have to catch the exception being thrown by the Filer and adjust project state accordingly. However, this approach produces a bug: consider the case when an invalid file name is passed. Filer produces the same error as when file is deleted, which could lead to incorrect behavior should oldSize variable be something else than 0. For now, this bug persists and it will be something for me to fix in the near future.

At the end of the day, my pull request is being reviewed as I am writing this blog post… Hopefully, they will read it after landing the PR. =)

Preparing for Release 0.2 in Mozilla Thimble

My first release for Mozilla’s Thimble taught me some invaluable aspects about Open Source contribution. Having experienced the whole pipe line of meaningful contribution to a large project, surfaced multiple crucial points of how to approach code examination.

My intention for the second release is to deeper my understanding of the back-end part of the system by working on introduction of a new feature. My issue talks about restricting user project size.

During the discussion on how this feature is supposed to be realized in the final product, it has become clear that I’ll have to implement a whole new module to Bramble. Have to admit, at first, I felt that I was over my head. However, examination of other Bramble modules has relieved the initial panic. It seems like, this module will need to trigger some kind of event on Bramble based on the current project state. Moreover, I have presumably located a bottle neck function that adds files to a project. My guess is I will invoke my module in that function to preform the check.

This is more or less everything I know right now of which I am at least partially confident. Hopefully, I’ll be able to proceed further.=)

Contributing to Thimble

Contributing to Open Source could be tricky and I’ve been able to experience how bad it gets over the last week. Choosing an Open Source project to contribute to was not a hard task, it was pretty much decided for us. Mozilla’s Thimble project is an open source online editor for people who are in the process of learning HTML, CSS, JavaScript. There were a bunch of issues labeled as “good first bug” and I thought it was the way to go. The actual bugs looked pretty much the same for me – I did have any clue of how to solve them. Eventually, I ended up picking this issue. The problem was that nav header was moving to a weird position when window was to narrow.

In the issue discussion, there has already been some thoughts on where this problem might be living. You see, Thimble projects actually consists of two sub-projects: Thimble project and Mozilla’s Brackets project. Brackets is the actual editor and Thimble is just a host for the iframe that contains Brackets. This  is done due to certain security concerns, since Brackets is running JavaScript this could lead to a potential breach. To avoid it, Brackets and Thimble are hosted on different services which eliminates a possibility of outside interference.

Installing and running Thimble was pretty easy thanks to this blog post. Here are a couple of screen shot of how Brackets and Thimble works.

Brackets (note names for editor parts in red):

brackets_with_labels

and Thimble (note nav headers of Thimble that are synchronized with Brackets):

thimble_with_labels

Initial comments suggested to look for the issue inside Thimble, so I’ve spent next couple of days looking in the wrong place. I think of it as a great insight on the system, since it gave me vague understanding of how Thimble works. Hours of stepping through the code landed me in Brackets, in the Bramble extension. It was a whole new portion of the system, so it took me an other day to finally find something useful. I’ve stumbled across a set of triggers that listened for layout updated. From there it was easy enough to locate the code that actually updates the layout.

At this point, I thought that my problem has finally appeared. Hours of code tinkering proved me wrong. So I sought help from projects developers. David Humphrey was really helpful in providing some valuable information regarding how modules interact with each other. His comments has also strengthened my suspicions of certain parts of the code. The follow-up talk with a friend, who was working on a new extension for Brackets, has cleared the rest of grey areas and allowed me to pin point the exact cause of the problem.

The issue was caused by the action sequence of how cached width for editor had been initialized. Brackets and Thimble use percentage-based system to update the layout, but it seems that cached values are stored as pixel values. Of course, this wasn’t my initial thought, one my initial pull request I’ve just deleted code that initialized the value of the editor. I admit, it wasn’t my brightest idea. After a couple of comments, I was able to come up with a proper formula for editor’s width initialization.

Thimble issue : https://github.com/mozilla/thimble.mozilla.org/issues/1154
Pull request : https://github.com/mozilla/brackets/pull/581