I think you can write any program using only advanced concepts from the problem domain, not advanced concepts from programming itself. For example, I'll happily implement a difficult numerical algorithm for inverting a matrix, but I won't write a monad framework in C++ for that. IMO there are almost no projects that call for programming cleverness (as opposed to problem domain cleverness). You can get very far with boring imperative code that any beginner can read, with powerful algorithms introduced only locally and documented clearly.
With some caveats. Execute 10 network requests in parallel on background threads, then reconcile their results, after that fire off another 3 requests in parallel (again, on a background thread), reconcile all the values you got so far, and return to the main thread. Oh and repeat the whole thing every time the user session's token changes, or the user requests a manual refresh. Also don't forget to cancel all running requests when the process is interrupted before it is finished. One more thing: If one request fails, bubble the error to the main thread and skip all subsequently planned requests.
You CAN solve this with counters and locks (be careful about deadlocks though), but you will save yourself a lot of pain if you just use a FRP framework. You will also be able to easily react to the things you forgot ("Oh the requests should also be halted and retried after connectivity is lost").
So where do you draw the line? Do you spare your coworkers the agony of learning FRP or do you implement something where you suspect it will horribly crash and burn in production?
For the past couple of years, I have been employed as a sysadmin, but for the past ~two years, my main task has been to implement the glue that hooks up several software systems we use and write what could be reporting tools.
Programming-wise it's been kind of dull, actually. No fancy clever stuff. At the same time it's been demanding in terms of understanding the business perspective on things. How do you define "costs accumulated on a given project" or "coverage contribution of a given project"? Having no background in business management or accounting, I have little to no clue, but it sure was interesting to find out.
I'll echo that sentiment, I experience it daily. Very rarely is a business itself abstract and complicated enough to merit an abstract and complicated technical solution. Meatspace is still pretty simple.
There's often a non-complicated solution to a difficult problem but spotting it takes some skill and experience and sometimes takes longer to implement than a more obvious complicated solution.
And sometimes complicated problems can really be solved only by a complicated solution, but can be half-assed in a simple way. You pay for the partial solution much later.
It is good to avoid future discounting in this way.
I rarely solve difficult programming challenges. I solve difficult business challenges all the time.
This is an expedient way of presenting difficulties encountered in software development. Suppose you need to collate information from many different processes that are distributed in space and time, possibly pursuing the information from a series of back-up sources when the primary source fails, and providing on demand, at any point along the way, a summary of results taking into account pending results and failures.
This is a straightforward problem for a human. A person does not need special training to pick up the phone when their boss calls and say, "The framer finished on Tuesday, the faucet supplier hasn't replied to any of the voice mails I left last week so I sent an email to the plumber since he's used them before, the electrician says the wiring won't pass inspection the way we asked him to do it, and the client approved a draw for $10k so there will be no problem paying for the tile on Friday." Someone who does not program computers does not see a need to analyze this problem and create a precise and detailed plan for how to pursue and summarize this kind information. The difficulty arises when attempting to do something like this with a computer.
Therefore, it's reasonable to describe it as a programming challenge. But you'd be foolish to. You're much better off calling it a business challenge, because getting it right matters to other people and the proper modifier for stuff that matters is "business." If it's "programming" then you're just amusing yourself on company time.
We've internalized this terminology to a sick degree. I took someone's dependent little brother to a diabetes appointment for them, and when I was summarizing the visit to her, I accidentally referred to the doctor's directions for taking the various medications as the "business logic." I meant to distinguish it from the things the doctor explained merely to satisfy my curiosity. As programmers we've had it drilled into us that "business" means something that matters to other people and can be discussed with them. Anything that is not "business" is at best mildly self-indulgent to mention and at worst selfish to even think about.
Therefore, we strive to describe everything we do as "business." This is how the statement
I rarely solve difficult programming challenges. I solve difficult business challenges all the time.
becomes a declaration of virtue.
As an illustration of how words affect how we think about things, replace the word "programming" with "engineering." We aren't ashamed of tackling "engineering" challenges. So there's an out when something is hard and matters but can't plausibly called a business challenge: call it an engineering challenge instead.
There are many tasks that are easy to solve with a human brain and very hard to solve with a computer. Anyone except a programmer will see them as sane and generally understood and will be satisfied with a description of the problem that would be sufficient for a human being. These are the kinds of problems where a programmer has to be very careful about the perception of his or her work.
The naive programmer sees that all of the difficulty has been introduced by the decision to solve the problem with a computer system. Therefore it is the programmer's responsibility to solve the difficult problem and justify the difficulty to others. Concurrency is not a problem for human beings, so if concurrency is a difficulty, the programmer must take the time to solve it, and if a non-programmer asks why he is taking so long, he must do his best to explain why concurrency is hard for computer programs.
The wise programmer shields himself by exposing as much of the difficulty as possible onto non-programmers without every mentioning the computer. Never mind that the task is well enough specified for a $15 an hour temp worker to carry out -- it is ridden with underspecified "business logic!" The naive programmer assumes that the system must operate under any reasonable ordering of events or failures. The wise programmer brings obscures scenarios up in meetings, suggests that such-and-such a scenario isn't valid, furrows his brow worriedly at the answer, and then proclaims that the "business rules" need to be "more completely specified" to handle these "previously out of scope cases."
Again, never mind that in the past a dozen different human beings with no special training performed the task reliably without needing to call meetings to find out how to do their job. Never mind that no difficulty existed until the introduction of a computer that needed to be programmed. The wise programmer understands that the task is difficult and he needs to make everyone else feel that difficulty so that he doesn't appear to be struggling with an easy assignment. Therefore a task that the business has been performing successfully since its inception must be found to be ridden with previously undiscovered "business challenges."
I'm not entirely sure whether I should be reading that as an earnest explanation regarding recurring problems with automation, or a gradually increasing level of sarcasm aimed towards lazy developers.
Neither developer is lazy. They both work hard and solve a hard problem. However, the naive developer is perceived as taking a long time on an easy problem for no good reason, and might not even be allowed to finish. The wise developer is perceived as taking a disappointing but reasonable amount of time on a surprisingly difficult problem. That the wise developer achieves this by acting disingenuously and wasting a lot of other people's time tells us something about a social phenomenon we see in the comments here, namely that programmers care quite a lot about how they label the difficulties they face, and they feel most comfortable with labels that classify the difficulty outside their area of professional competence.
Obviously the absurdity bothers me. I didn't enjoy being the naive developer, and I don't think I would enjoy being the wise developer, either, if I were capable of it. That means I'm doomed never to work alone. Right now my team is eight months into a project that was supposed to produce a prototype in three months, and we are yet to produce anything beyond a handful of disconnected demos. If I was doing this project by myself, or if I was leading a team of people like me, we would have been relieved of responsibility a long time ago. But we're fine because we have a couple of wise developers who are capable of communicating the difficulties in the manner described above, by essentially teaching other parts of the company that they don't know how to do their jobs.
I dislike the reality and am intentionally painting an unflattering picture of it, but I'm not trying to assign blame or disparage any party, because I don't have a solution. I just thought I'd present it as a way of seeing things.
My career in software development started with BASIC - Pick BASIC to be exact. It was 1992, and I was 18 years old, had moved to Phoenix a week after graduating from high school to attend a local tech school (now defunct), and I needed something to pay the rent after having my hours cut as a cashier for Osco Drugs.
Seeing my other classmates at the time taking "apprenticeship" positions doing electronics work, I decided to "cold-call" nearby companies, and landed on a "mom-n-pop" shop doing software development work. I put in my application, and they took a chance on me. They didn't hire me as a software developer, though - but as an "operator" - the guy to "run reports, keep paper in the printers, and mount tapes" for the actual programmers, to free up their time. I learned how to load a open-reel 9-track tape drive with vacuum columns at that place...something I have never used since.
I was "in heaven" - being paid much more than I was as a cashier (but finding out much later that I was actually "underpaid" - oh, the ignorance of youth!), while doing less work! They gave me access to an account on their IBM RS6000 system running AIX - and one of the guys gave me a book on PICK BASIC to look over, and play around with.
The shop developed "insurance claims management" software for the "Arizona Health Care Cost Containment System" (AHCCCS) - basically Arizona's form of Medicaid; this company's software managed claims (accounting, billing, etc). They used a form of PICK called "UniVerse":
PICK (and UniVerse) were actually very interesting systems on their own; kinda a self-contained OS that integrated a programming language (BASIC) with a relational (and multi-value!) DB, along with other stuff. It compiled to a byte-code, which some manufacturers made CPUs to run (instead of interpreters); the idea was "portable software" across multiple architectures. Basically the idea of Java before Java. It worked out about as well, of course (because every implementation of PICK was just a bit different).
Anyhow - the programming language used was BASIC - and my experience up to that point was writing bits and bobs of BASIC (plus some assembler - for the 6809 and 6502 processors - interestingly the latter was for the Apple IIe - CALL -151 was my friend; hand assembly of op-codes in the monitor - ugh). Well - they were monitoring the account, as I coded up adventure games and other diversions in PICK BASIC. The programmers and owner started feeding me with tasks ("hey, can you fix this issue" or "can you build this demo for the printer", etc) - and I proved myself. They hired me after I had graduated the tech school, and I've been here in Phoenix ever since.
As far as BASIC and businesses are concerned - it is still used today, as much as everybody rants on it. Besides Visual Basic (large following there), PICK BASIC is still used (it has a legacy almost as long and broad as COBOL), and there are many other old "business BASICs" that existed and still have users today. I also know that there is at least one industrial robot arm manufacturer who sells a "Robot BASIC" to program and control their arms.
...and BASIC lives on today, of course - both closed and open-source variants. I'm more familiar with the various open-source BASICs. You have Gambas (which looks very much like VB), then there is QB64 (multi-platform 64 bit compiled QBASIC if you will), and also something called "BaCON", which is a "BASIC-to-C" converter (just the other day I was thinking taking it, and dumping out some Web Assembly code - essentially to see if I could get a BASIC program compiled to run in a web browser). There are also more than a few interpreters in javascript and other languages that can run BASIC code of some sort or another.
Would I recommend it for new projects of large scope? Probably not, but I wouldn't outright discount it, either. I think it still has its place in the computing world, though that world is doing its best to ignore it in the hope that it is forgotten. It did teach some "bad habits" - but I think that a modern form of it, with proper OOP support, could be a very powerful teaching language for beginners - which is what it was always meant to be.
It will always have a spot near and dear to my heart, at least.
Thanks for letting us into your lovely personal computing story.
I often ruminate about the graveyard of dying software - the BASICs, the COBOLs, the xBases, the Delphis and the PowerBuilders, to name just the few I know. And the immense amount of code written and souls poured into by programmers over the years.
They're all there, there is still juice in them, but we move on because, impermanence. This was also a theme in _why's parting gift. https://news.ycombinator.com/item?id=5575707
This is exactly the attitude I try get into every developers head!
Please write maintainable code using common and broadly understood methods. There is no award for surprising me in how fancy your solution is, in fact the opposite.
"Please write maintainable code using common and broadly understood methods."
Often times, the most common methods are the most obfuscated and unclear.
Maybe this is my perspective just because I am a Java developer. Pulling in lots of poorly understood dependencies. Depending on annotations that change execution flow in complex ways that are very difficult to follow, even with a sophisticated IDE. Deploying into complex web containers. ORMs mapping method calls into who knows what SQL calls. Lots of superfluous mutable state everywhere.
(I'm looking at you Spring. Although Spring Boot is a definite improvement.)
Sorry, the use of the word 'common' was a poor choice [and I'm actually not sure what would be a better choice]. I agree with you wholeheartedly. I honestly find a lot of those common methods very confusing.
While I do not go around trying to beat people with a stick when they make things I see as overly complex, I think it's good to have a conversation about it, sit down together to find the simpler solution [or accept the one that is already implemented] and hopefully reach alignment on the path forward.
It's totally possible my knowledge is lacking, which is why the conversation is critical. If I just need to be educated on why something I saw as complex was the right decision, so be it, and I look forward / enjoy those opportunities.
At the end of the day, I run the tech team and founded this company, while others might come and go, I need to feel ownership over the code, product and most importantly it's shortcomings. If I cannot own that openly, I cannot lead/direct others, and that is when technical teams really start to fall apart imo.
The only thing that lays further ahead on that road is the lowest common denominator, copy-pasted code and lots and lots of mediocre code monkeys. I don't want to go there, although I can understand people who would like programming to work that way.
Sure, but to my mind monads are common and broadly understood, at least compared to all the alternatives for achieving the same kind of thing (e.g. AOP shudder).
> but to my mind monads are common and broadly understood
Wow, you need to talk to more developers. Not just in the sense to broaden your views, but to improve your skills as a developer. Code is written to be read by humans (and orthogonally executed by machines). If your mental model of what is broadly understood by programmers is that far off, you are likely churning out lots of unmaintainable code.
Functional programming makes up a tiny slice of the programming world and most people that aren't doing functional programming (and even some who are) don't have a strong grasp of monads.
But isn't typical high-quality OOP code essentially Monadic code? State is encapsulated in an object and transformed by method calls. Method calls can be chained, and state can be encapsulated and extracted using constructors and accessors, respectively. The basic operations are all the same. OOP is just less strict with how you can transform objects, i.e. function/method purity is not enforced.
> But isn't typical high-quality OOP code essentially Monadic code?
Nope.
> State is encapsulated in an object and transformed by method calls.
Monads aren't objects (which necessarily have a runtime manifestation). Monads (as used in functional programming) are abstract type constructors (which normally don't have a runtime manifestation).
> Method calls can be chained, and state can be encapsulated and extracted using constructors and accessors, respectively.
Monadic bind satisfies three concrete, clearly defined algebraic laws. The use of monads in functional programming arose from the field of denotational semantics, which gives “programs” (by which functional language semanticists mean “expressions”) precise mathematical meanings in a compositional manner.
I understand what a Monad is. My point is that high-quality OOP code typically adheres to the rules of Monads.
I didn't say Monads were objects. I simply talked through the most common properties and operations of OOP code, in order to show how they are closely related to the operations on Monads.
I understand that OOP code does not, in general, satisfy the constraints of Monads. Hence, why I specified high-quality OOP code, which, in contrast, typically does.
> My point is that high-quality OOP code typically adheres to the rules of Monads.
Last I checked, there are only three laws: left/right identity and associativity, and they are just algebraic laws that have nothing to do with your code being “good” or “bad”. Or do you have some other meaning of “rules of monads” in mind?
Furthermore, every higher-order strict language is already monadic, without you having to do anything special. Thus, monads aren't opt-in, or even opt-out; they're there whether you like it or not. This monadic structure permeates the language, regardless of whether you write “good” or “bad” programs in it. For more info, see here: https://news.ycombinator.com/item?id=13823145
Yes, those are the laws. The theorems are more interesting from a maintaining-software perspective, though.
Yes, all higher-order strict languages can be embedded in a monadic model. However, there's an important practical difference between a single Monad with the entire application or computer's state encapsulated (course grain encapsulation), and many Monads with fine-grained encapsulation. High-quality OOP code can be embedded in the latter, and that's where the maintainability comes from.
---
Perhaps, if you focused more on understanding the point I'm trying to make instead of proving me wrong, this conversation would be more beneficial to both of us.
Computation in every higher-order, strict language has an intrinsic monadic structure. That includes pretty much every object-oriented language in existence. (Ironically, it doesn't include Haskell.)
And, by "intrinsic", do you mean that "the language does that, but the programmer doesn't have to do anything to make it happen, or even know"? If so, doesn't this contradict your answer to smaddox?
I imagine the GP is talking about how code in procedural languages is equivalent to monadic code on pure languages.
If so, I don't know why the "higher-order" restriction is there. It seems unneeded. Also, it about is as much something "the programmer doesn't have to do anything or even know" as monadic code on languages that have specialized syntactic sugar, like Haskell. It creates plenty of details the programmer has to think about, the only difference is that in procedural languages all the details are always there, while in pure ones the detail set changes all the time.
Procedures as first-class values that you can store in variables, pass as arguments to other procedures, etc.
> And, by "intrinsic", do you mean that "the language does that, but the programmer doesn't have to do anything to make it happen, or even know"?
Yes. Although, IMO, programmers benefit from knowing it, for more or less the same reasons civil engineers benefit from knowing the laws of nature.
> If so, doesn't this contradict your answer to smaddox?
No. Monads, as abstract type constructors, are already hardcoded into the result type of every computation. (This is the difference between, say, an int like “2”, and a computation that produces an int like “1 + 1” or “fix (\x -> x + 1)”.) So hardcoded, in fact, that there doesn't have to be syntax for it, because there's no way you can not say it.
Will you get upset if someone in your java codebase uses lambdas or optionals or even the ternary operator? All of the above are still considered black magic to many in our industry.
If a standard feature in your language of choice confuses or upsets you when it is not misused... You may have a problem. The problem is typically lack of willingness to learn.
It totally depends on the situation, is it something the whole team needs to work with on a regular basis? And is it really the most optimal solution (those two questions are highly interlinked)?
If that's the case then we might want to a) educate everyone else first or b) pick something we know will be less of a struggle for others to maintain.
I guess there is always option c) ask the original author never to leave and be available 24/7 to answer questions, but that, obviously, isn't realistic.
Edit: I answered the question poorly. Are you "writing art" or doing your job? I don't mind if the outcome is pretty, but your first priority is to do your best in your role within the organization. If the author or the code feels that is the case then generally I won't ever get upset with anything they do.
I work on a product that was mostly C. At the time the developers loved it! Look at how clear and explicit everything is. These days however people recoil in horror when they see parts still in that old C style.
The rest of the world can and will change their definition of what "common and broadly understood" means.
OP's point is that the best way to write maintainable code is using bog standard, boring code and leaving clever solutions to the level of the problem domain.
Bog-standard meaning not testable and not contract-based?
Clever solutions and abstractions (hey, immutability is clever right?) can really help out reasoning.
And sometimes you really need clever things due to problem domain, so you really end up with totally not boring code.
For example, you wouldn't cook up your own matrix manipulation everywhere, so you use a clever solution instead. Writing operations directly opposed to composing and optimizing them.
Or in high performance world, using a much more complex but lock-free algorithm instead of the "bog-standard" locked one, sometimes the benefit is huge.
The real important part is to exactly document the clever parts and enforce contracts on them, so you do not end up with potpourri everywhere.
I find myself getting rid of everything clever, my code reads like a boring instruction manual.
I'm starting to deeply re-think how code interviews should be conducted.
Instead of asking smarty-pants questions, we should be asking mediocre questions and just asking someone to put together a straight-forward and simple solution.
Important: it's surprisingly difficult to make code that looks simple and boring :), it often takes some iteration.
I don't think you can write maintainable code without monads. Code has to be written in the language of the business domain so that it can be understood in those terms, so you can't (to take an extreme example) have every other line being "if err != nil ...". But if you leave secondary concerns completely implicit then your code also becomes unmaintainable because you can never understand what any given function is doing (at the implementation level) and so you can never maintain anything. You need the dual perspective that monads or something like them give you where you can say "this is a secondary concern in this code but it is there".
It's not about "programming cleverness" - there's nothing "clever" about monads, rather the monad is a very simple and boring interface that turns out to be a good tool for helping you write code that does what it looks like.
What does maintainable mean? I propose a definition: we can add features at a constant rate, but the rate of bugs per line of code drops off over time. Linux certainly doesn't meet that threshold: the rate of bad bugs is pretty constant. If we decompose it, we'll see a safe maintainable section with a monadic use of goto, and a less safe section that doesn't.
NASA's projects that do hit this standard tend to be written in CL with macros that do this.
Thats an interesting model, but I cant think of many projects where the features are added at a constant rate. The deluge of pull requests going to Linux are not constant, and probably grows with the number of contributors perhaps relative to population growth of programmers (also not constant).
Perhaps another metric is how long after your code is released will you be able to support it? In which case Linux and NASA do quite well.
I would love to learn what a nomad is. Can you explain what it is to someone who already is programming for many years? I just find the Wikipedia article very hard
A nomad is a wanderer. A monad is just a particular abstract interface that turns out to come up a lot and so lead to very reusable code. It's hard to talk about monads in general because the concept is very abstract, I find the best thing is to understand a couple of specific examples first (Writer and Future are good simple cases, and then Nondeterminism or Continuation gives you some idea of the powerful things you can do) and then try to work up from there to the abstract part that unifies all these things and lets you write general-purpose functions that will work with any of them.
http://m50d.github.io/2013/01/16/generic-contexts is an explanation of a case where I found myself using the Applicative interface - Monad is very similar to Applicative (it just adds one extra possible operation).
In short they are way to automate building new types out of old types in such a way that code written for the old types can immediately be turned into code written for the new types. This allows a lot of otherwise repetitive code that you might otherwise wind up writing to be pushed to your type system. Which in turn makes it feasible to build a richer type system.
