What is the formula to find the number of diagonals in a polygon?

Answer 1

Suppose the polygon's vertices are denoted A1A2...An. We can fix one endpoint of the diagonal at A1 and count the number of ways to choose from the remaining vertices. There are n-3 vertices to choose from, because we cannot count A1, nor count A2 or An (since they are directly next to A1).

There are n vertices total, so we multiply to get n(n-3). However, we have to divide by 2 because we count each diagonal twice. The diagonal AiAj is the same as AjAi, and each is counted. Therefore, the number of diagonals in an n-gon is n(n-3)/2.