Remake.js: Part 1

I started working on a web framework last summer. I didn’t know I was working on a framework though.

It started out with just a simple positioning library. I wanted to be able to position popovers. Then I added another library that would handle setting element dimensions on popovers automatically.

Then, I added the big one, or what I thought was the big one: Widget.js.

Widget.js (later renamed Switch.js) had been used in many projects at that point, including the original version of Artisfy.

What did it do?

It’s very simple, really. It would let you click a button to activate a widget. If you clicked inside that widget, you could interact with it, but, if you clicked outside of it, it would close.

Very simple stuff. It basically was a very, very short piece of code that stood in for code that I would write over and over and over again.

But then, it got more complicated.

Sometimes I wanted to click somewhere else and have the widget close, but sometimes I didn’t want it to close. Sometimes I wanted to activate multiple widgets at the same time.

Anyways, it turned into a big piece of code that’s actually really elegant in what it does.

This, however, didn’t constitute a framework. All I had was the ability to activate or deactivate some widgets on the page and position them.

It only turned into a framework when I started thinking about DATA.

It started out simply enough.

I had this beautiful idea for a web framework:

What if I could make a webpage, edit that webpage, and then just save the edited webpage?

Isn’t this how all web apps work though?

Well, yes, but this one took things to a whole different level of literalness. Most web apps let you edit DATA and then save the DATA — and then display that data in a visible format.

But, what I was thinking was:

What if I just bundle up everything together and save it all at once? Then there would just be one source of truth: the webpage.

If you’re not understanding yet, this is how it’d work:

1. Load up a webpage
2. Click a button to edit some area on the webpage
3. Edit the text in that area and press save
4. The ENTIRE webpage’s code would be saved to the server

But… this is insane… isn’t it?

Well, not really. It’d let you make simple, interactive webpages very, very quickly. All you would need is HTML, CSS, and the ability to change the HTML. No database, no schema, no back and forth with the server. All the data in on the webpage and it’s all saved. What the user sees is what they get the next time around.

You’d only want to give people access who you trusted, of couse, because they could add ANY code they wanted, including external JavaScript that was meant to steal passwords.

But, even that, could be prevented. You could use a content security policy on modern browsers to make sure no external scripts were added. And you could take the extra precaution of stripping out dangerous code with DOMPurify or something.

The elegance of this solution was beautiful to me. As a front-end developer who doesn’t like messing around with databases and sessions, it was perfect for creating quick, one-off projects.

It was as simple as: “Edit HTML, Save HTML.”

What about authentication? Well, the server could handle that.

And when you get an admin user, you add certain classes to the HTML that show them the edit controls.

And, when you get a regular user, you strip all the admin classes out of the document.

It was pretty sleek.

… But I ran into a problem: What about using data in more than one place?

What if I have a webpage with a username and then on the very next page I want to display the same username? Do I force the user to change it twice? That’s very unintuitive… People have gotten used to thinking in terms of settings and preferences that are persistant and global. People don’t really think that webpages are each just separate documents.

But! There’s a solution for that too! You could use the webpage as a database and query it. So, if you’re looking for the username, just keep track of its element’s selector and pass that into your template… Then you could search through other pages’ data using a document query syntax, just like you’re working on the front-end.

However, it’s not that simple. Here’s why:

1. At some point you’ve got to wonder what you’re doing with your life if you’re slowly reinventing something like PHP from scratch. PHP isn’t bad, but it’s already out there. And it has support for templates and stuff out of the box. If I’m slowly going to be creating something different, but not better, and gradually adding on features until it’s comparable to something that already exists… what am I doing?
2. You’d need a single source of truth for your data. So, if you have a username and you want to use it in multiple places and you’re going to query a webpage for it… you need that webpage you’re querying to be the only source of truth and the other pages to be dependent on it.

Of course, there are ways around the 2nd point.

And the 1st point is like… whatever. It’s still nice to have new things in the world even if other things exist that can do the same thing. You never know if the new thing will be better in some way… eventually.

Anyways, though, I decided against implementing it because it sounded like too much work for me… especially back-end work, with lots of data and complicated configurations and security implications. Stuff I’d rather not be responsible for at the end of the day. The risk of a mistake is too high.

However, it’s still a good idea, especially for lightweight demos:

1. Edit a page on the front-end
2. Save the resulting webpage
3. Strip out any admin/active classes between page loads

Boom! Web framework!

So, I think this is something I’d like to work on eventually, some day.

But, let’s get back to my journey to creating a framework.

So, here I was, feeling a little despondent that my idea of saving a while webpage wouldn’t work, especially because of the whole “how do you reference state from two different places?” problem.

And… I came up with an idea.

At the end of the day, I’d be rendering back-end templates anyways, which support variables and interpolation and stuff. So, why not just find a way to markup my pages, so I could quickly extract the data from them… and then feed this into the back-end templates?

I know… I know… I’m not saying anything new again. This is how ALL web apps work… haha.

But, it was a little bit different… because I was thinking about making it automatic, so you wouldn’t even notice… So, it would be almost as simple as just saving a webpage.

Anyways, without further ado, this is what I came up with:

<div data-o-type="list">
	<div data-o-type="object" data-o-key-name="David"></div>
	<div data-o-type="object" data-o-key-name="John"></div>
	<div data-o-type="object" data-o-key-name="Sam"></div>
</div>

Making up the HTML with these data-o-type attributes would allow a custom parser I’d write to quickly and easily extract the data from the DOM that was necessary to render it again…

Basically, it would capture everything about a webpage that the server needed to render that page.

If you’re curious about what the o in the data-o stands for, it’s “output”. That’s because it’s data that’s being output from the DOM, as opposed to inputted by the user (we’ll get to that later).

So, let’s talk about what the above piece of HTML would look like as data.

First, we define a list with data-o-type="list".

Then, inside of that are some objects, with data-o-type="object" on them.

That means we’ve got some objects inside a list.

Pretty easy, right?

The key thing to notice is how the structure of the data corresponded to the structure of the DOM. If there’s a nested DOM node, it most likely belongs inside of the DOM node above it. Or, at least, at the same level. So, that’s what this library (Remake.js) assumes.

Here’s what the above elements look like as actual data:

[
	{name: "David"},
	{name: "John"},
	{name: "Sam"}
]

And, here’s what the back-end template that would render this might look like:

div(data-o-type="list")
  each val in ["David", "John", "Sam"]
    div(data-o-type="object" data-o-key-name=val)

(This is pugjs btw).

How useful is though? I mean you’re storing data in the attribute, but it’s not on the actual page… It’s not displayed anywhere…

Well, in order to address that, we have data-l-key-* attributes, which, by default, look for their value in the innerText of the current element (but can be told to look elsewhere).

So, now, instead of the original HTML example, you could use something like this:

<div data-o-type="list">
	<div data-o-type="object" data-l-key-name>David</div>
	<div data-o-type="object" data-l-key-name>John</div>
	<div data-o-type="object" data-l-key-name>Sam</div>
</div>

And this will give you the exact same output:

[
	{name: "David"},
	{name: "John"},
	{name: "Sam"}
]

Which you can feed into your back-end template, which now looks like this:

div(data-o-type="list")
  each val in ["David", "John", "Sam"]
    div(data-o-type="object" data-o-key-name)=val

And that’s it!

Now you have the ability to write some HTML, attach a few data attributes that make that HTML serializeable, so you can extract the data from your HTML, and feed it back into your templates. It’s a really lightweight framework… kind of.

:)

Next time we’ll talk about some features I’ve been thinking about building. Also, we’ve barely scratched the surface of this project and its capabilities. In fact, I build an entire application using this framework which is in production right now.

And, while it does have its faults (it’s getting a little hard to put all the pieces together sometimes and it does some hacky things like storing all data as strings…), it’s been a real pleasure to work with and made it 100% faster for me to build new features.

However, the next features I’m building for it are meant to address some of the problems I’ve run into when thinking about adding themes to my application.

I’ll talk more about that in the next post.