© 2021 IEEE.ModSecurity is an open-source web application firewall. The correct configuration is key to its effective operation but could be tricky since usability flaws could lead to errors and misconfiguration, threatening the security and availability. In this study, we first conduct the usability inspection of ModSecurity through a hybrid approach consisting of heuristic evaluation and cognitive walkthrough. We find out that ModSecurity has no feedback for semantic errors or successful rule implementation during the rule entry process, and the webserver goes down if there is a syntax error in any implemented rule. We propose a rule checking and feedback mechanism to mitigate these drawbacks. Then, we perform a between-subjects user study to evaluate the effectiveness and usability of our proposed method. The results indicate that unsuccessful rule entry rate and syntax error-related web server downtime significantly reduced with the rule checking and feedback mechanism introduced to ModSecurity. Thereby, we improve ModSecurity's effectiveness as well as the level of security and availability of ModSecurity-protected web servers.