Free Memory refers to the amount of unused memory on a database instance. Free memory will normally decline over time as the database process and OS takes up as much memory as they are configured to use.
A low amount of free memory can cause the OS to start swapping memory in and out of swap space. Reading from swap space is significantly slower than reading from memory and can greatly slow down the database engine. When your instance is low on free memory, check its swap usage as well to see if it is swapping. If it is, then you should either re-configure the amount of memory used by the database, or increase the instance size.
Increasing the instance size is the most straightforward solution, but comes with extra cost to run the database. Depending on your RDS set up, changing the instance size could also cause a service disruption.
The database engines supported by RDS are highly tunable and will have configuration options to control how much memory to cache sizes, buffer pools, and how many connections can be allowed to the database -- all of which affect memory usage. These are provided through the RDS resource called DB Parameter Groups. Some of the default values provided by AWS use calculations that depend on the instance size, and it may be appropriate to either manually set the options, or increase instance size. Depending on your database engine, changes to a DB Parameter Group may require restarting the database instance, which may cause a service disruption.