I wrote a short tutorial on using my implementation of ASDL. You can find link to it in the document.
I don’t have a blog or things like that so I just used a Github Gist.
For some reason I got locked out of my Github account and if it was not for a miracle I would have been locked out forever. I barely use Github anyways. I don’t understand why Tarvolds uses this shit corporate hellhole of a website as the primary platform for his projects, which sadly includes Git.
I made a model in ChatGPT to walk me through libgit2 so I can make my own remote host. I don’t like any of the other bloated Git on Web platforms that are out there. I know about Git Tea, but that damn thing is bloated. I want some thing like this guy has: http://c9x.me/
This is what I mean when I say ‘OWN YOUR DATA’. Don’t trust Github, or any of these corporations, with shit.
I get this is really hypocritical to use Gist, then turn around and call Github a soulless corporation. But sadly, I use it BECAUSE it’s a soulless corporation, not in spite of it.
I don’t have money to buy a server. Yes, I am that poor. That is why I have to trust this PaaS and SaaS assholes.
Anyways fuck Github.
Congratulations on taking a step towards self-hosting and meta-circular compilation. ASDL is a great intermediate meta-language and it can be used to abstract any sort of Builder or Command workflow. This is sometimes called a “narrow waist”; ASDL replaces ad-hoc AST-builder objects with a unified protocol and interface.
For example, I encoded Monte into ASDL while rewriting the compiler in Monte, as part of a self-hosting effort. I also wrote a module which parses ASDL and emits Monte modules, including AST-building tools at runtime. Monte supports E-style quasiliterals, including source-code literals. This let our compiler directly consume ASDL files and emit Monte source code. Going beyond the compiler, this allowed me to encode UI elements and widgets as ASTs and use the Command pattern to write widget-handling objects.
Thanks a lot my good man. Is this Monte? https://github.com/monte-language/monte I can’t find the source code?
Can you tell me what you think of the code my implementation emits when you got the time? Is it good, bad, medicore etc?
So when you’re building a language, should you always use a builder for the AST specifications? Becuase I figure, you don’t need that in a language like Haskell or OCaml right?
I currently have 3 projects I ping-pong.
1- Marsh, a POSIX shell in C; 2- Awk2X; a translator of AWK to several languages, like C, Python Rust etc. This one is in Haskell 3- Cephyr; a C compiler in OCaml
For Marsh, I am stuck in the job control section. I just can’t seem to find a flow for redirecting input and output. I think I am striking a balance though. I revised the job control like 10 times, literally 10 times.
But for Awk2X and Cephyr, I am kinda stuck at defining the AST. I learned Haskell just last night and I made this AST. I am not ‘stuck’ really, I will move on from it soon.
data UnaryOp = Prefix PrefixOp | Postfix PostfixOp deriving (Show) data PrefixOp = PrefixIncr | PrefixDecr | Plus | Minus | Not deriving (Show) data PostfixOp = PostfixIncr | PostfixDecr deriving (Show) data BinaryOp = Add | Sub | Mul | Div | Mod | Eq | Ne | Gt | Ge | Le | Lt | And | Or deriving (Show) data Lvalue = Unfixed String | Fixed String String deriving (Show) data Factor = Constant Constant | Lvalue Lvalue deriving (Show) data Constant = String String | Integer Int | Float Float | Regex String deriving (Show) data Expr = Unary UnaryOp Factor | Binary Factor BinaryOp Factor | Ternary Expr Expr Expr | StrCat [Expr] | Delete Expr | Call String [Expr] | Assign Lvalue Expr deriving (Show) data Stmt = Express Expr | For Expr Expr Expr Stmt | ForIn Lvalue Lvalue Stmt | DoWhile Stmt Expr | While Expr Stmt | If Expr Stmt [(Expr, Stmt)] (Maybe (Expr, Stmt)) | Return (Maybe Expr) | Break | Continue | Compound [Stmt] deriving (Show) data Pattern = Begin | End | Expr Expr | ExprPair Expr Expr data PatternAction = WithPattern Pattern [Stmt] | JustAction [Stmt] data FunctionDefn = FunctionDefn { name :: String, params :: [String], body :: [Stmt] } data Element = PattAct PatternAction | FnDef FunctionDefn newtype Program = Program [Element
Now for Cephyr, this one is a bit more complex. Before Cephyr I attempted to make a C compiler several times, and every time I got stuck at the AST.
The problem is, I wanna use OCaml’s modules in the AST. But I don’t know how? I just said fuck it and used top-level types. I am here currently:
type const_expr type type_decl type unary_op type binary_op type expr type stmt type literal = ConstLiteral of const_lit | CompoundLiteral of compound_lit and const_lit = | IntConst of int * int_suffix option | FloatCOnst of float * float_suffix option | CharConst of char * charset_prefix option | StrConst of string * charset_prefix option | IdentConst of string * const_expr and int_suffix = Long | LongLong | Unsigned | UnsignedLong | UnsignedLongLOng and float_suffix = Double | LongDouble and charset_prefix = Local | U8 | U16 | U32 and compound_lit = | ArrayCompound of init_type * init_item list | StructCompound of init_type * init_item list | UnionCompound of init_type * init_item list and init_item = | RegularInit of expr | DesignatedInit of designated_init * expr and designated_init = | ConstExpr of const_expr | ConstExprPair of const_expr * const_expr | Ident of string and init_type = type_decl type const_expr = | UnaryConstExpr of { factor: literal; op: unary_op; } | BinaryConstExpr of { left_factor: literal; right_factor: literal; op: binary_op; } type primary_factor = Identifier of string | NestedExpr of expr | Literal of literal and expr = | Primary of primary_factor | Unary of { factor: primary_factor; op: unary_op; } | Subscript of { factor: primary_factor; op: subscript_op; } | Binary of { left_factor: expr; right_factor: expr; op: binary_op; } | Ternary of { cond: expr; if_true: expr; if_false: expr; } | Assignment of { lvalue: expr; rvalue: expr; op: assign_op; } and prefix_unary_op = | Negate | LogicalNegate | BitwiseNot | AddressOf | Dereference | PreIncrement | PreDecrement and postfix_unary_op = | PostIncrement | PostDecrement and unary_op = | PrefixOp of prefix_unary_op | PostfixOp of postfix_unary_op and subscript_op = | Index of expr | DotMember of expr | ArrowMember of expr | FunctionCall of expr list and binary_op = | Add | Subtract | Multiply | Divide | Modulo | BitwiseAnd | BitwiseOr | BitwiseXor | ShiftLeft | ShiftRight | LogicalAnd | LogicalOr | Equal | NotEqual | LessThan | LessThanOrEqual | GreaterThan | GreaterThanOrEqual and assign_op = | Assignment | AddAssign | SubtractAssign | MultiplyAssign | DivideAssign | ModuloAssign | AndAssign | OrAssign | XorAssign | ShiftLeftAssign | ShiftRightAssign
It’s still incomplete. I just found out I can use
.mli
files.I think Cephyr is the 5th reincaation of my OCaml C compiler. I just spend hours at the AST and get tired of it.
I found lcc, by Fraiser et al:
And I have the book too. I like the book but it’s kinda useless for me because I wanna do SSA. These kinda tree-rewriting mumbo jumbo is too 80s for my taste.
So any help is appreciated. Thanks.
Your code looked alright. Working in C is a risky chore. You’re early in your journey and I think it’s good to get a taste of many of the traditional techniques before turning towards newer fancier algorithms.
“Haskell and OCaml”, hm? The two concepts you need to internalize are katamorphisms and paramorphisms. You may have heard of “recursion schemes”; these two schemes are the ones available on any tree. Fundamentally, most of a compiler is tree-to-tree transformations (nanopasses), and they are expressible as one of two forms:
- Katamorphism: The leaves of each node are transformed before the branch (bottom-up)
- Paramorphism: The leaves are transformed after/during the branch transformation (top-down)
If you look again at my AST builder builder, you’ll see
.run()
and.walk()
methods, implementing the recursion for any katamorphism or paramorphism respectively. In Haskell, these are calledTraversable
types. This is a pun in Monte, where.run()
is also the default method; the syntax makes it easy to perform a katamorphism by passing a tree-traversing object directly to the AST.Your types are alright, but you’ll want to pass a generic parameter through them, turning them into a valid
Functor
in Haskell or a generic module in OCaml. This is a basic defense against the “AST Decoration Problem”, AKA the “AST Typing Problem”. As you add features to your compiler, you’ll realize why these are necessary.(Sorry if this is a double post) I think what you call ‘decoration’ I call 'augmentation;. After many iterations ,I came up with this AST: https://pastebin.com/NA1DhZF2
I like it, but I am still skeptical as of comitting to it. I really want to use Functors… But i’m nto sure how. These are new concepts to me!