It starts with “Why complexity exists? and is it important?”. It exists because of mathematics. Let me cook, in general terms when we here the term “complexity”? We here it for the first time in our Data Structures and Algorithms course. So one thing is clear that it it related to Algorithms because Data Structures are used to store data there is no rocket science in that. Now to generalize Algorithms I can call all the algorithms “programs” because that’s what it is. The work of a program is to take input and produce some output, so if i try to reason on how can work of a program grow, the only option is to increase the size of my input. How does one track “how much has the work program grown?” because there is nothing to measure it. This led to the birth of complexity which measures the growth pattern of a program under huge load of work. So where does the mathematics part ? Here : when we try to compute complexity of any program, it will be dependent on that machine, which will be inaccurate. Why, because my decade old samsung laptop can’t compete with someone’s latest macbook pro with latest chip and highest specs. So what computer scientist did is they made this concept machine-independent (laptop, desktop or servers). There are some basic operations in every program like variables, comparisions, loops etc. We can just count these on pen and paper and calculate complexity of any program. It is just really a theoretical lens.

Evolving to Time Complexity

Remember I talked about growth of work of program, and as the work grows it is certain that the time taken by program will also grow. So the first question comes to my mind is why do we calculate time? Simple answer is computers have limited time(CPUs can do only so many steps per second) then next question arises how do we calculate this time? In seconds? Like “this particular program took 2 seconds to complete this task”. But it isn’t going to work because same principle: every machine is different, every programming is different, CPUs is going to evolve so this time is definitely going to reduce. So seconds are unreliable. We need something machine-independent. This thinking leads to Time Complexity.

Flipping to Space Complexity

Time matters but what about memory(in computer science whenever we talk about memory, we are referring to RAM), just like time computers have limited memory. In a similar fashion to time complexity, when we think of memory for a program, this leads to Space complexity, which is definately not calculated in megabytes(MBs).

Why Big-O was chosen ?

Since i was studying this topic, I came to know that there isn’t any formula for time and space complexity, and it seems counter intuitive to me because through out life if something comes under the umbrella of maths it should have formula to calculate it. But that is not the case for complexity. When I dig deeper, I found some problems:

• Exact formula depends on machine: Which is definitely not acceptable since we discussed it already that we need something machine-independent. For instance, let’s say that you wrote a program(Algorithm) and it takes 3n + 5 operations on your laptop. Ask yourself a question “what is one operation?” is it one cpu cycle, one line of code or one assembly instruction. Different machine answer differently. So its not going to take 3n + 5 operations on my laptop, it could be more or less.

Constant stop mattering as input grows: Suppose I have written 2 algorithms “Algo A” and “Algo B”. Algo A takes 3n+10 operations and Algo B takes n2 operations. For very large n, which one explodes? when n = 10:
A ⇒ 40
B ⇒ 100

but when n = 1,000,000

A ⇒ 3,000,010

B ⇒ 1,000,000,000,000

So in this case +10 and x3 doesn’t matter anymore. The actual thing we care about is “How does this algorithm behaves when input becomes huge?” not exact time, exact steps, exact constants. So to avoid all these pain computer scientist introduced Big-O which can be calculated on pen and paper. It stays stable, is machine-independent and answers growth rate. And when we compare 2 different algorithms that’s where it helps. Big-O gives a rough idea on performance of algorithms.

Conclusion

We are:

• abstracting away details

• focusing on growth

• modeling behavior mathematically

Hope I was able to add value to your today’s learning goal !! Happy Learning ..