In this conceptual paper we argue that it is impossible to sustain crop productivity in the long run under a complete weed eradication program because of the loss of above ground and rhizosphere biodiversity which will ultimately reduce crop productivity even under the most aggressive use of weed resistant crops, agrochemicals as fertilizers, insecticides, herbicides, and irrigation water. Instead a sustainable crop production with moderate yield can be achieved with a better understanding of the cropland ecology and managing the population biology of major weeds. Allelopathy can play an important role in achieving this objective along with other non-chemical weed control measures. The challenges and opportunities of ecological management of weeds will be highlighted with examples.