## **Computational complexity theory** ### **From Wikipedia, the free encyclopedia** Jump to: __navigation__, __search__ In __computer science__, **computational complexity theory** is the branch of the __theory of computation__ that studies the resources, or *cost*, of the computation required to solve a given __computational problem__. This cost is usually measured in terms of abstract parameters such as time and space, called __computational resources__. *Time* represents the number of steps it takes to solve a problem and *space* represents the quantity of information storage required or how much memory it takes. There are often tradeoffs between time and space that have to be considered when trying to solve a computational problem. It often turns out that an alternative __algorithm__ will require less time but more space (or vice versa) to solve a given problem. Time requirements sometimes must be amortized to determine the time cost for a well defined average case. Space requirements can be profiled over time, too, especially in consideration of a multi-user computer system. Other resources can also be considered, such as how many __parallel processors__ are needed to solve a problem in parallel. In this case, "parallelizable time" and "non-parallelizable time" are considered. The latter is important in real-time applications, and it gives a limit to how far the computation can be parallelized. Some steps must be done sequentially because they depend on the results of previous steps. Complexity theory differs from __computability theory__, which deals with whether a problem can be solved at all, regardless of the resources required. ## **Overview** After the theory explaining which problems can be solved and which cannot be, it was natural to ask about the relative computational difficulty of __computable functions__. This is the subject matter of computational complexity. A single "problem" is an entire set of related questions, where each question is a finite-length __string__. For example, the problem __FACTORIZE__ is: given an integer written in __binary__, return all of the __prime__ factors of that number. A particular question is called an *instance*. For example, "give the factors of the number 15" is one instance of the *FACTORIZE* problem. The **time complexity** of a problem is the number of steps that it takes to solve an instance of the problem as a function of the __size of the input__ (usually measured in bits), using the most efficient __algorithm__. To understand this intuitively, consider the example of an instance that is *n* bits long that can be solved in *n*І steps. In this example we say the problem has a time complexity of *n*І. Of course, the exact number of steps will depend on exactly what machine or language is being used. To avoid that problem, we generally use __Big O notation__. If a problem has time complexity O(*n*І) on one typical computer, then it will also have complexity O(*n*Іp(n)) on most other computers for some __polynomial__ p(n), so this notation allows us to generalize away from the details of a particular computer. **Example:** Mowing grass has linear complexity because it takes double the time to mow double the area. However, looking up something in a dictionary has only logarithmic complexity because a double sized dictionary only has to be opened one time more (e.g. exactly in the middle - then the problem is reduced to the half). [__edit__] ^ Much of complexity theory deals with decision problems. A __decision problem__ is a problem where the answer is always YES/NO. For example, the problem *IS-PRIME* is: given an integer written in binary, return whether it is a prime number or not. A decision problem is equivalent to a *language*, which is a set of finite-length strings. For a given decision problem, the equivalent language is the set of all strings for which the answer is YES. Decision problems are often considered because an arbitrary problem can always be reduced to a decision problem. For example, the problem *HAS-FACTOR* is: given integers *n* and *k* written in binary, return whether *n* has any prime factors less than *k*. If we can solve *HAS-FACTOR* with a certain amount of resources, then we can use that solution to solve *FACTORIZE* without much more resources. Just do a __binary search__ on *k* until you find the smallest factor of *n*. Then divide out that factor, and repeat until you find all the factors. Complexity theory often makes a distinction between YES answers and NO answers. For example, the set NP is defined as the set of problems where the YES instances can be checked "quickly" (i.e. in polynomial time). The set Co-NP is the set of problems where the NO instances can be checked quickly. The "Co" in the name stands for "complement". The __complement__ of a problem is one where all the YES and NO answers are swapped, such as *IS-COMPOSITE* for *IS-PRIME*. An important result in complexity theory is the fact that no matter how hard a problem can get (i.e. how much time and space resources it requires), there will always be even harder problems. For time complexity, this is determined by the __time hierarchy theorem__. A similar __space hierarchy theorem__ can also be derived. [__edit__] |