I Designed A Programing Language. And Here Is My Experience.
For about two weeks, I've been designing a programing language called Scrpt Engine, Even though its slow and definitely not bug free, It was an amazing experience learning about how compilers and interpreters work.
The goal was to make a very easy to use language with core programing features such as: Variables, Functions, Classes, Statements, loops, Imports and more. With its most unique feature, being able to write other languages inside the same file.
How it all started
It was about 7 months ago I first started coding, before then ive just been using drag and drops. My learning motivation only came when I finally realized I've been overthinking how difficult programing was. I finally got the motivation and started learned Java. It was my Computer science class in high school and since the language seemed pretty fun, it was my go to language. Since then I've been trying to work on image processing and Artificial intelligence.
After a while, Java started becoming harder to use, It wasn't getting enough love on the field I was going for, so I got stuck. Me who was trying not to be the beginner programmer, who learned multiple languages eventually switched to Python. And I will say, it was well worth it, Python had all the AI modules and easy syntax I needed.
Since then I've just been experimenting with python, Benchmarking, Working with Neat-Python, Pygame, Kera's and a lot more.
How I Started Building the Language
I will first start by saying, this did not take me one attempt, It came by multiple attempts trying out many other methods of building just one language with so many hours of YouTube grinding.
My First Attempt
My first attempt was simple, convert the text into a python representation and use exec to execute the code. I was working on that idea for a while, then discovered there is absolutely no reason to do that, The goal is to improve on my programing skills. Using exec was just not the best route to follow.
My Second Attempt
My second attempt was to actually compile the language into some byte code representation ready to be executed by my self built byte code compiler and finally be executed. But for someone who was just 5 months into coding, What do I know about byte code? I eventually discarded the idea and moved on
My Third Attempt
This is the idea that finally worked for me, I took a far more simpler approach. Since Python is a dynamically typed language and I can use that to my advantage here is how I went about.
The Platform
If you are going to make any kind of Programming language, you have to build it with a language too. if you are going to interprete the language then C++ becomes a great tool to use because of its incredible performance. Python can great if performance is not your main goal or you are building a compiled language. I also wanted to build something that wouldn't give me way too much head aches so this is how I used python to my advantage.
Squishify
A programing language can quickly be broken down into a more compressed representation just by removing un-needed spaces but there are also spaces you should not remove, You cant get rid of spaces in strings or statements and since the file might contain literal blocks of code, you definitely cant remove them. This is something that just had to be planned. Indented code also have to be grouped to the parent control flow statement or function they follow. With all that, we can make a compressed representation of the code
Lexer/Tokenizer
This groups the raw text into a collection of groups of text with the type they are and maybe some extra information for example “x=24” becomes {variable:x},{operator:”=”},{int Literal:”24”}
Parsing
This finally structures those tokens and give them order and eventually use. for example:{variable:x},{operator:”=”},{int Literal:”24”} becomes: x which should be an integer pointer should now point to address of 24; or assign into value of x to 24.
Lambda/Python Variables
This is what made my programing language much easier to create. It relies on python variables. And as depressing as that sounds stores variables in a dict, That obviously was a very slow approach. Plus building a dynamically typed language on another dynamically typed language means runtime will run snail speed.
Even though Speed shouldn't be in my mind if am making such decisions but I have to care. I have to do a lot of pre processing, Giving variables memory address and scope before runtime which also have to work for classes and recursion before runtime.
The goal was to compile the text into python objects in which every object as a .run function. and a .__init__ function. .__init__ gets executed before runtime, while parsing, this gives variables life and does some more.
.run happens at runtime, my goal to making this fast is to put anything that will not change during runtime in the __init__ function.
and with all that the program finally runs, and at least, is not 30 times slower than python itself.
By far the hardest part in building this was memory, Statically giving variables addresses and Making it bug free, This will have been so much easier if python has pointers (I also tried making the same thing in C++, It did not end well)
Conclusion
My goal was not to flex on building a language but to motivate people who have interest in coding in general, It took me time to discover programing is just as hard as you think it is. It only becomes a skill set when you are constantly learning and if you have something you are planning on building just build it, work on it, It may take sometime to finally implement Eventually you will finally achieve it.
And as of now the Scrpt Engine works fine, you can do stuff with it, probably make a small game with it. But there is still a lot to work on, Error messages come from python, you cant index lists, surprisingly no else statement and some random bugs. Am still planning on releasing it on github soon
