This research explores the challenges in management and the root cause for complex application portfolios in the public sector. It takes Australian public sector organisations with the case of South Australia Police (SAPOL) for evaluation it being one of the significant and mission critical state government agencies. The exploratory research surfaces some of the key challenges using interview as primary data collection source, along with archive records, documentation, and direct observation as secondary sources. This paper reports on the information analysed surfacing eight key issues. It highlights that the organic growth of the technology portfolios, with mission criticality has resulted in many quick fixes which are not aligned with long term enterprise architectural stability. Integration of different mismatched technologies, along with the pressure from the business to always keep the lights on, does not provide the opportunity for the portfolios to be rationalised in an ongoing way. Other issues and the areas for further study are explored at the end.