CSC/ECE 506 Spring 2013/8c da

2013-03-20T23:56:15Z

Dthomas:

'''Snoop Filters'''

==Introduction==

One of the issues with large systems with multiple processors having shared memory and each processor having its own private cache is the cache coherence problem. The non-coherent view of values of a single data item in these different caches is referred to as the cache coherence problem. A protocol which ensures a coherent view of cached values as seen by multiple processors is referred to as cache coherence protocol.

Snooping is the process where the individual caches monitor address lines for accesses to memory locations that they have cached. When a write operation is observed to a location that a cache has a copy of, the cache controller invalidates its own copy of the snooped memory location. Many cache coherence protocols require a hardware component to monitor the bus transactions.

Hardware support for implementing cache coherence protocols at each node in a bus-based multiprocessor system can be provided using coherence controller. The coherence controller has a component called the snooper. The role of the snooper is to snoop each bus transaction involved in the cache coherence transaction. For each snooped bus transaction, the coherence controller checks the cache tag array to see if it has the block that is involved in the transaction, checks the current state of the block (if the block is found), and reacts accordingly by responding with data or by changing the state of the block.<ref>Fundamentals of Prallel Computer Architecture by Yan Solihin</ref>

In most well-optimized programs, much data is not shared among threads. So most of the time, most snooped bus transactions do not find the block in the local cache. Even in that case, snooper snooped the bus transaction and and checked the cache tag to determine whether the cache has the block, thus incurred unnecessary work. There is the possibility that contention can occur between processor and the snooper to access the cache tag. One possible solution to reduce contention between the processor and the snooper is to introduce a snoop filter, which determines whether a snooper needs to check the cache tag or not. By reducing the number of snooped transactions the need to check the cache tags, contention and power consumption can be reduced.

==Why Snoop Filtering?==

With the advent of modern computers built with multiple processing cores and shared memory programming model it has become necessary to use cache coherence protocols to maintain coherence between different caches attached to individual processing units and many coherence protocols are snoop based.

===Challenges faced by Snoop-based protocols===

Snoop protocols face three main challenges:

'''1. Request ordering:'''

It is necessary to ensure the relative order of snoop broadcasts. This can be achieved in small multiprocessors by relying on network ordering properties. But larger multiprocessors with arbitrary network topologies cannot rely on the network to order requests.

'''2. Network bandwidth requirements:'''

Snoop-based protocols require to do snoop broadcasts during all cache misses.The snoops consume large amounts of network bandwidth and this is the main factor that limits the scalability of snoop coherence protocols.

'''3. Tag look-up bandwidth requirements:'''

In addition to network bandwidth, each broadcast consumes significant tag look-up bandwidth as each cache checks to see if it has a copy of the requested block. As the number of cores in the system grows, the rate of snoop-induced tag look-ups can cause contention that delays demand look-ups from the local core and hurts performance.In addition the challenge of simply providing sufficient network and tag look-up bandwidth, it is important to consider the energy consumed in the network and in the tag arrays as a result of snoop broadcasts.

===Snoop Filter as a solution===

Each bus segment can be separated into different cache coherency domains using a snoop filter, with very little traffic occurring between the two. The snoop filter is most likely implemented as a large table that stores recent cache line requests, the state (MESI) of each cache line, and bits to indicate which segment the cache line is in (or both). When a cache miss occurs, the originating CPU broadcasts a snoop request on its bus. Both the snoop filter and the other CPU in the package will receive the request and take action appropriately. If the read request hits in the snoop filter, then it will check where the requested cache line is located. If the requested cache line is only available on the other bus segment, then the snoop request will be sent to the other segment. If the requested cache line is available on both buses or only on the originating CPU’s bus or only in main memory, then the snoop filter does not pass along the request, thereby saving front side bus bandwidth.

A snoop filter is a small cache-like structure present between the data cache and the bus as shown in figure, which filters probable data cache misses and forwards those snoop induced look-ups that might result in a hit. The snoop filters achieve filtering by either keeping track of a super-set of blocks currently present in the cache or those that are not present in the cache.
The most important factor to be considered while designing these filters is that a look-up in any of these filters must consume less energy than a normal cache look-up. Several snoop filter architectures have been proposed in previous literature depending on the filtering methodology.

One way to implement the snoop filter is to duplicate the cache tag array and filter out all snoop requests that miss. But this is not practical and can result in serious performance bottlenecks. But, it has been shown that very accurate filtering can be achieved with small designs that conservatively approximate the cache contents and operate at a reasonable frequency.

==Types of Snoop Filters==

There are many different attributes that can be used to categorize these filters. We chose to use the point of origin as the first order attribute for classification. These filters can be grouped into three broad classes:

===Destination Based Snoop Filters===

Destination-based snoop filters are the ones those are inclined to reduce the number of tag look-ups as a result of various snoop broadcasts. These in effect don’t reduce the number of snoop broadcasts; however they try to optimize the actions taken in response to it. They achieve this by filtering the snoop requests and hence avoid local tag look-ups. This optimization technique tries to reduce energy and bandwidth utilization.

On receiving a snoop request, a these category of snoop filters might give off immediate response without requiring a tag array look-up. This goes in to reduce the energy consumption and also the contention for the tag array. For general purpose large multiprocessors, these tag look-ups can easily out-number demand look-ups from the local processor, giving way to high contention and a loss of performance so it is important to try to reduce that contention.

====Atoofian and Baniasadi Filter====

This is a kind of filter needing very little area and energy overhead. It maintains a table consisting of saturating counters at each cache with one counter for each processor core in the system. When a snoop request is received for reading, it only performs a lookup if the counter corresponding to the requesting core is “saturated”. If not so, it replays with an negative acknowledgement. There might be another core with a positive reply or if the requester may get all negative acknowledgements. In that case, it tries again and this all of them are force to do look-ups. A “saturating” counter is always incremented when a cache supplies data in response to a snoop request from the corresponding core, and it is reset to zero when a snoop request from the corresponding core fails to find the data in the cache. This protocol serves good in case of workloads that exhibit supplier locality. They reduce energy and contention traffic. However if the workload does not exhibit supplier locality, this may have significant performance losses. An example of such filter is SPLAS-2.

====Inclusive Filters====

Inclusive filters keep a track of all lines that result in positive responses to snoop requests. In other words, an inclusive snoop filter keeps track of a super-set of the blocks that are cached. This ensures a miss in an inclusive filter is guaranteed to miss in the cache, so there is no need to forward the request. Similarly, a hit in an inclusive snoop filter may or may not hit in the cache, so the request must be forwarded. The pivotal challenge here is to provide an area and energy-efficient structure that can represent this set of blocks.

=====Subset Filters=====

'''Strauss et al.'''

Strauss et al. filters focus on filtering snoops for read requests and track those blocks that are in a “supplier” state and hence could give a positive response to a snoop request. A filter by the name of “subset” filter keeps track of the subset of blocks in the cache that are in one of the “supplier” states. As a result of which lines in shared state will return a negative reply to a read snoop. A set-associative array contains the tags of all supplier blocks in the cache. This at times might result in increased bandwidth utilization and also the subset filter only keeps track of lines in order to do efficient tag-look up for read snoop requests only and it does filter the invalidate messages.

=====Superset Filters=====

'''Stream Register Snoop Filters'''
This was introduced by IBM Researchers which went on to be used the Blue Gene/P supercomputer. This uses stream registers to encode cache lines stored in the cache in particular way as mentioned in the following text. Each of these stream register (SR) is consists of the following; a base register, a mask register, and a valid bit. The base register here keeps the starting point of a line to be reached out, while the mask register encodes the entries of the block that have been accessed as offsets of the base. Due to space constraints, the offsets are not represented explicitly. Rather, the mask represents a super-set of the offsets that have been accessed.

'''Counting Stream Register Snoop Filters'''

The disadvantage with the above filter is over time, as more and more unique addresses are accessed by the filter, more and more of the bits in the mask will be set to 0. Hence, all possible addresses that the stream register can filter will decrease over time. Gradually, all of the mask bits become zero, and the SR filters no further addresses. At this point in time, even if the address that were accessed long back are being evicted out or invalidated then also they will not be filtered buy the snoop filter. This happens until a cache wrap occurs and all of them are flushed out to start afresh. The important point is this whole flushing part results in significant amount of overheads.

Counting Stream Register Snoop Filters overcome this by keeping a counter value instead of the valid bit used in the original Stream Register based Snoop Filters. Each time a particular block offset address in accessed, in addition to making the corresponding super-set of bits to 0, it also increments the counter value by 1. And every time a cache invalidation or eviction occurs, it decrements the counter value by 1. So by using this logic, the overhead of flushing out the register in case of cache wrap is not required. It simply checks the counter value and filters it if the value is 0.

====Exclusive Filters====

In contrast to the inclusive filters mentioned above, an exclusive snoop filter maintains information about blocks that are not being cached. A hit here ensures that the cache does not contain the block, so there is no need to forward the request. However a miss must be forwarded to the cache for processing.

=====Exclude-Jetty Filters=====

This is an exclusive filter where a set-associative table containing the most recently snooped addresses that returned negative responses is maintained. This technique benefits due to the principle of temporal locality in snoop addresses, filtering out most unnecessary snoops for highly contended blocks. The Blue Gene/P system incorporates a variation of this in the name of “vector exclude Jetty” as part of its snoop filter.

=====Blue Gene/P Range Filter=====

The Blue Gene/P has a different kind of exclusive snoop filter namely “range filter”. This filter keeps track of a range of addresses that are either outside the range of the pertinent cache or they are not cached. Hence the snoop requests for this range of addresses are ignored. The range filter is software-controlled and performs well when multiple processors are known to be using completely distinct and continuous portions of physical memory.

===Source-based Snoop Filters===

The destination-based snoop filters filter out unnecessary tag lookups but these requests and their corresponding responses still require bandwidth. Hence if the requesters can somehow able to predict in advance either that no other caches will have copies of the requested block, or that only certain cache might have copies, then it can avoid sending a snoop broadcast all together, or potentially send a multicast instead of a full broadcast. This is the technique used by so called source-based snoop filters. This also results in decreased bandwidth utilization and better scaling snoop protocol to many cores.

====Speculative Selective Requests-based Snoop Filters====

Speculative selective requests (SSR) uses the idea of keeping a “saturating counter” and a supplier ID information at each core to predict the supplier cache from where to ask required block when needed. It starts incrementing the counter corresponding to a particular core every time a block is provided by the same core. When this counter reaches a saturating value, the cache controller further approaches this pertinent supplier directly for the block when needed. Until then it uses the broadcast mechanism. For workloads that exhibit a large degree of supplier locality, such as SPLASH-2, data is often streamed from one cache to another, making SSR very effective. However, other workloads with less predictable behavior see little benefit from SSR as the counter's seldom saturate, or potentially suffer performance losses if frequent miss predictions occur.

====Software-Assisted Filters====

Various software based source-based snoop filters that avoided broadcasting snoops for certain accesses where those snoops were known to fail were also proposed. The first filter is based on the fact that data in the stack of each thread are private and are not shared hence snoop request for these need not be sent. Another proposed filter mechanism uses a “Snoop-Me-Not” bit to all instructions that access memory. Programmers, compilers and operating systems aid in setting the bits appropriately depending on the type of application. This technique requires minimal architectural support, but needs the programmer or compiler to understand the memory layout of the program, and possibly deal with issues.

===Other Categories filters===

====Serial Snooping====
Serial-snooping is a form of protocol-based filtering where snoop messages are exchanged between involved processors until a valid copy of the data is found. In this way, it reduces the number of messages exchanged. However this may not be the case always. The benefits of serial snooping also largely depend on the network topology in which it is being used. In some cases latency may be reduced while in other cases this may lead to unnecessary overheads.

====In-Network Snoop Filtering====

Consider small networks with each network comprising of a set of caches. The networks here are interconnected by routers. The idea used in In-Network Snoop Filtering is to modify the outgoing snoop broadcasts from a network depending on some predefined criteria and the information regarding the networks where to send the request are stored in a table at each outgoing router of a network. A RegionTracker cache is used for the same which tracks which regions are present in each cache. In other words, each router uses a small table to track the sharing information for the most recently accessed regions. Hence a snoop broadcast is only sent to those regions who actually share the relevant block. The major setback in this approach is that in-network filtering apparently is not compatible with non-adaptive routing protocols.

====Snoop Filtering in Virtualized Multi-Cores====

In virtual environments, usually the sharing tends to occur only between threads and processes running in the same virtual machine. Hence by the virtue of this boundary, snoop requests could only be limited to within this virtual machine rather sending it everywhere. However such an approach requires the support of the operating system along with hyper-visor.

==References==
<references />

CSC/ECE 506 Spring 2013/8c da

2013-03-20T23:04:21Z

Dthomas:

CSC/ECE 506 Spring 2013/8c da

2013-03-20T22:58:59Z

Dthomas: Created page with "=Snoop Filters= ==Introduction== One of the issues with large systems with multiple processors having shared memory and each processor having its own private cache is the cache..."

=Snoop Filters=

==Introduction==

One of the issues with large systems with multiple processors having shared memory and each processor having its own private cache is the cache coherence problem. The non-coherent view of values of a single data item in these different caches is referred to as the cache coherence problem. A protocol which ensures a coherent view of cached values as seen by multiple processors is referred to as cache coherence protocol.

Snooping is the process where the individual caches monitor address lines for accesses to memory locations that they have cached. When a write operation is observed to a location that a cache has a copy of, the cache controller invalidates its own copy of the snooped memory location. Many cache coherence protocols require a hardware component to monitor the bus transactions.

Hardware support for implementing cache coherence protocols in a bus-based multiprocessor system at each node can be provided using coherence controller. The coherence controller has a component called the snooper. The role of the snooper is to snoop each bus transaction involved in the cache coherence transaction. For each snooped bus transaction, the coherence controller checks the cache tag array to see if it has the block that is involved in the transaction, checks the current state of the block (if the block is found), and reacts accordingly by responding with data or by changing the state of the block.

In most well-optimized programs, much data is not shared among threads. So most of the time, most snooped bus transactions do not find the block in the local cache. Even in that case, snooper snooped the bus transaction and and checked the cache tag to determine whether the cache has the block, thus incurred unnecessary work. There is the possibility that contention can occur between processor and the snooper to access the cache tag. One possible solution to reduce contention between the processor and the snooper is to introduce a snoop filter, which determines whether a snooper needs to check the cache tag or not. By reducing the number of snooped transactions the need to check the cache tags, contention and power consumption can be reduced.

==Why Snoop Filtering?==

With the advent of modern computers built with multiple processing cores and shared memory programming model it has become necessary to use cache coherence protocols to maintain coherence between different caches attached to individual processing units and many coherence protocols are snoop based.

ECE506 Main Page

2013-03-20T22:48:47Z

Dthomas:

This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.

=Supplements to Solihin Text=

Post links to the textbook supplements in this section.
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]
*[[CSC/ECE 506 Spring 2012/1a ry]]
*[[CSC/ECE 506 Spring 2012/1c dm]]
*[[CSC/ECE 506 Spring 2012/1c cl]]
*[[CSC/ECE 506 Spring 2012/1a mw]]
*[[CSC/ECE 506 Spring 2012/3a yw]]
*[[CSC/ECE 506 Spring 2012/7b yw]]
*[[CSC/ECE 506 Spring 2012/3b sk]]
*[[CSC/ECE 506 Spring 2012/4b rs]]
*[[CSC/ECE 506 Spring 2012/6b am]]
*[[CSC/ECE 506 Spring 2012/8a cj]]
*[[CSC/ECE 506 Spring 2012/10a dr]]
*[[CSC/ECE 506 Spring 2012/10a jp]]
*[[CSC/ECE 506 Spring 2012/9a ms]]
*[[CSC/ECE 506 Spring 2012/10b sr]]
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]
*[[CSC/ECE 506 Spring 2012/12b jh]]
*[[CSC/ECE 506 Spring 2010/8a fu]]
*[[CSC/ECE 506 Spring 2010/8a sk]]
*[[CSC/ECE 506 Spring 2012/11a ht]]
*[[CSC/ECE 506 Spring 2013/1b dj]]
*[[CSC/ECE 506 Spring 2013/1a sp]]
*[[CSC/ECE 506 Spring 2013/1d ks]]
*[[CSC/ECE 506 Spring 2013/2b so]]
*[[CSC/ECE 506 Spring 2013/1c ad]]
*[[CSC/ECE 506 Spring 2013/3b xz]]
*[[CSC/ECE_506_Spring_2013/4a_aj]]
*[[CSC/ECE_506_Spring_2013/4a_ss]]
*[[CSC/ECE_506_Spring_2013/1a_ag]]
* Chapter 3a [[CSC/ECE_506_Spring_2013/3a_bs]]
* Chapter 6a [[CSC/ECE_506_Spring_2013/6a_cs]]
* Chapter 5a [[CSC/ECE_506_Spring_2013/5a_ks]]
* Chapter 8a [[CSC/ECE_506_Spring_2013/8a_an]]
* Chapter 7a [[CSC/ECE_506_Spring_2013/7a_bs]]
* Chapter 8b [[CSC/ECE_506_Spring_2013/8b_ap]]
* Chpater 8c [[CSC/ECE_506_Spring_2013/8c_da]]

2012-10-03T20:45:52Z

Dthomas:

'''Active Records'''

==Introduction==

The ''Active Record pattern'' is a Design pattern in Software Engineering which deals with the approach to store and access data in a database. The interface of an object conforming to this pattern would contain functions to perform operations like Insert, Read, Update, and Delete. The Object will have properties that correspond to the columns in the underlying database table. This pattern is realised through ORM (Object-Relational Mapping) libraries in Programming languages.

''ActiveRecord'' is a module for Ruby that can be used for ORM. Thus, it is obvious that ActiveRecord will form a part of the Model in an MVC application developed in Ruby. The rest of the article discusses ActiveRecord that is the Ruby module for implementing the Active Record pattern.

The ActiveRecord module insulates the developer from the need to use SQL in most cases. Internally, It will perform queries on the database which corresponds to the method invoked on the object. This module is compatible with most database systems (most used ones like MySQL, PostgreSQL and SQLite). Moreover, regardless of which database system the developer uses, the Active Record method format always remains the same.

==Naming==

The ActiveRecord module uses a convention for naming classes, tables and fields so that the amount of configuration needed to get the functionality working is minimal. There are naming conventions on file naming, class naming, table naming etc.

===Reserved names and Attributes===

Certain names are reserved and should not be used (even in the model as attributes). Some of them are listed below:

*lock_version.
*type - This is only used when you have single table inheritance and must contain a class name.
*id - Reserved for primary keys.
*table_name_count - Reserved for counter cache.
*position - Reserved for acts_as_list.
*parent_id - Reserved for acts_as_tree.
*lft - Reserved for acts_as_nested_set.
*rgt - Reserved for acts_as_nested_set.
*quote - Method in ActiveRecord::Base which is used to quote SQL.
*template.

===Class Naming===

ActiveRecord classes are named in singular form. e.g User

===Table Naming===

Tables for ActiveRecord objects are named in plural form by default. e.g. Users
This naming convention can be circumvented by using below:

a) Set use_pluralization
In the environment.rb file we can specify
ActiveRecord::Base.use_pluralization = false.

This will disable pluralization for all ActiveRecord objects.

b.) Use set_table_name
You can call set_table_name to specify a custom table name for a particular model.
For example:
class User < ActiveRecord::Base
set_table_name 'user'
end

== CRUD ==

CRUD stands for 'Create', 'Read' , 'Update' and 'Delete'. These are the four basic operations which are generally performed on tables in a database. The ActiveRecord module provides predefined methods for the basic CRUD operations for the model.

3.1 Create

A new record can be created in the table by invoking the “save” function on the model object whose record has to be created in the database. ActiveRecord will use the Object's attributes as the field values for the record. The data is not persisted in the database until we call the save function.

@user = User.new
@user.name = “abc”
@user.age = 23
@user.save #returns a boolean indicating whether the save was successful or not (whether a new record was created or not)

ActiveRecord provides another convinient way to create a new record without creating instantiating the model explicitly and then using “save”. To do this, we use the 'create' function. By default the primary key used in the table is “id” which is generated automatically.

User.create(:name=>”xyz”, :age=”23”)

3.2 Read

A record can be read from the table by using the various functions like “find” (find the model record by specifying a value used in its primary key), “where”, “all” , “first” and “last”. All these functions instantiate a new Object for the model and populate its attributes using the fields of the record.

@user_first = User.first #Finds and returns the 1st User from the table
@user_last = User.last #Finds and returns the Last User from the table
@all_users = User.all # Returns all the Users from the table
@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_other_user = User.where(:name=>”abc”) #Finds and returns the user whose “name” is “abc” .

Dynamic Finders

Some of the most common searchs performed on databases are to return the rows where a column matches a given value. In many other languages and frameworks, we would generally need to construct SQL queries to perform these searches. ActiveRecord uses Ruby’s dynamic power to do this for us.
[edit] Connecting to the Database
For example, our User model has attributes such as name and age. We can use these names in finder methods to return rows where the corresponding columns match some value:
@my_user = User.find_by_name(“abc”)
@my_user = User.find_by_age(15)

3.3 Update

A record in the table corresponding to a given model instance can be Updated by using the function “save”.

@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_user.name = “test”
@my_user.save

Moreover, we can combine the functions of reading a row and updating it using the class methods update and update_all. The update method takes an id parameter and a set of attributes. It fetches the corresponding row, updates the given attributes, saves the result to the database, and returns the model object.

@my_user = User.update(1, :name=”test3”)
@result = User.update_all(“age= age+1”)

3.4 Delete

A record can be deleted from the table by invoking the “destroy” functionality on the instance of the object.
The destroy instance method deletes from the database the row corresponding to a particular model object. It then freezes the contents of that object, preventing future changes to the attributes.

@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_user.destroy # deletes the record corresponding to the user with id = 5 from the table
# ... my_user is now frozen

It also has two class-level methods, delete and delete_all. The delete method takes a single id or an array of ids and deletes the corresponding row(s) in the underlying table. delete_all deletes rows matching a given condition.
User.delete(1)
User.delete([2,3,4,5])
User.delete_all(["age < ?" , 18])

The “delete” methods bypass the ActiveRecord callback and validation functions that may be defined for the model class, while the “destroy” methods ensure that they are all invoked. Hence, it is better to use the “destroy” methods as it ensures that our database is as per the business rules defined in the model.

==Connecting to the Database==

The ActiveRecord connection adapter is meant to wrap and abstract away the underlying driver specific to database, and is meant to provide an interface which is common for database tasks such as creating and destroying databases, modifying tables, updating, deleting, and inserting data, managing transactions and running queries . The connection adapter is normally used internally by ActiveRecord but can be used without the help of ActiveRecord models as well.

A connection adapter can be obtained in the following manner:

connection = Category.connection
object = Category.find(1)
connection = object.connection

Most applications connect to only one database which is defined in the database.yml file. In such a scenario every class which inherits from ActiveRecord::Base will be using the same connection. But in some special cases the application may also connect to a secondary database. That is the case in which some ActiveRecord classes connect to a secondary database. In such cases extra care need to take so that every class asks for a connection from the right database.

Rails generally opens several connections at once, and these connections are managed in a pool. Each connection adapter object forms a single connection to some database. Connections can run only one SQL statement at a time, so generally one connection is opened per thread. When a job needs a connection to database, it checks out one of the pool which is returned when it finishes for use by another task.

===Running Low-Level Queries===

ActiveRecord “model” objects are returned by ost calls in the standard ActiveRecord API. There might be cases in which you want to bypass the overhead involved in creating full ActiveRecord objects, or maybe want to query data that does not have a corresponding ActiveRecord class. SQL queries can be written using the connection adapter's low-level query methods.

In this first example, we get the “category_name” value from a single row in our “categoriess” table. If we only need the category name, we can grab the connection object and use the select_value method as shown below:

connection = Category.connection
category_name = connection.select_value("SELECT name FROM categories WHERE id=1")
# => "Football"

==Migrations==

Migrations help to version the various changes made to databases. It also allow developers to track a set of changes made to production or development databases and to rollback to a previous version if needed.

=== Building a Migration ===
You can either build the migration on its own using

<pre>ruby script/generate migration User</pre>

Specific commands can be written afterwards to create custom SQL. A model can also be created that comes with the migration.

<pre>ruby script/generate model User name:string user_id:integer</pre>

The migration will generate a couple of new files under the "db" directory. The contents of such a generate file are as follows:

# 9889904091223123_create_user.rb
class CreateUsers < ActiveRecord::Migration
def self.up
create_table :users do |t|
t.string :name
t.integer :user_id
t.timestamps
end
end

def self.down
drop_table :users
end
end

The "9889904091223123" at the beginning of the file-name is the timestamp. The timestamps will be different depending on the times of creation or modification the database. This is helpful to rollback to a previous state if needed. This way the developer need not remember how the current state is reached and how can he go back to a previous state.

The ''self.up'' from the previous code snippet creates the User table and add the columns. The ''self.down'' method is used to drop the table and to remove all the contents. The ''self.up'' and ''self.down'' methods are necessary to keep the database consistent after a rollback.

Rails adds an additional column called the "timestamps" to keep track of when each row was added. Rails also creates a primary key of the form "model_name"_id which increments automatically every time a row is added.

Different types of datatypes can be used with ActiveRecord. Some of the most commonly used ones are:

* integer
* string
* text
* boolean
* references
* decimal
* timestamp

The migration file can be written into the database using the following command:

<pre>rake db:migrate</pre>

The above command will create the table and the various columns. This command can be used to migrate many files in one go. Connections between tables in the database can be introduced using references in the model.

==Associations==

Associations are used to connect 2 models. The association is used to describe the role of relations that models are having with each other. ActiveRecord associations can be used to describe one-to-one (1:1), one-to-many (1:n) and many-to-many (n:m) relationships between models. Associations are used to make common operations simpler and easier in your code. Rails supports six types of associations:

* belongs_to
* has_one
* has_many
* has_many :through
* has_one :through
* has_and_belongs_to_many

''belongs_to'' and ''has_one'' form a one-to-one relationship. ''has_one :through'' is a different way to create a one-to-one relationship. ''has_many'' and ''belongs_to'' form a one-to-many relation. ''has_and_belongs_to_many'' or an alternative way ''has_many :through'' to create a many-to-many relationship.

===belongs_to Association===

A ''belongs_to'' association sets up a one-to-one connection with another model, such that each instance of the declaring model “belongs to” one instance of the other model. A ''belongs_to'' association can be used to setup a one-to-one or one-to-many relationship with other models. For example, consider a cookbook with recipes and categories such that each recipe "belongs to" a particular category.

class Recipe < ActiveRecord::Base
belongs_to :category
end

===has_one Association===

A ''has_one'' association is used to set up a one-to-one connection with another model such that each instance of a model contains one instance of another model. For example if we have two models User and Account, and each User has a single account, then we can use "has_one" to indicate the relationship between the two models.

class User < ActiveRecord::Base
has_one: account
end

===has_many Association===

A ''has_many'' association is used to set up a one-to-may association with other models such that each instance has zero or more instances of another model. In the cookbook example, one category can have many recipes.

class Category < ActiveRecord::Base
has_many :recipes
end

===has_many :through Association===

A ''has_many :through'' model is used to setup a many-to-many association with another model ''through'' a third model. In this case, the instance of a model can be connected to many instances of another model by proceeding through a third model. For example, consider a medical practice where patients make appointments to see physicians.

class Physician < ActiveRecord::Base
has_many :appointments
has_many :patients, :through => :appointments
end

class Appointment < ActiveRecord::Base
belongs_to :physician
belongs_to :patient
end

class Patient < ActiveRecord::Base
has_many :appointments
has_many :physicians, :through => :appointments
end

===has_one :through Association===

A ''has_one :through'' model is used to setup a one-to-one connection with another model ''through'' a third model. In this case, the instance of a model can be connected to one instance of another model through a third model. For example, each client has one account, and each account has one account history.

class Client < ActiveRecord::Base
has_one :account
has_one :account_history, :through => :account
end

class Account < ActiveRecord::Base
belongs_to :client
has_one :account_history
end

class AccountHistory < ActiveRecord::Base
belongs_to :account
end

===has_and_belongs_to_many Association===

A ''has_and_belongs_to_many'' association creates a many-to-many connection with another model without any model in between. In the cookbook example with recipe and category models, if the recipe is allowed to be in more than one category then the ''has_and_belongs_to_many'' association can be used.

class Recipe < ActiveRecord::Base
has_and_belongs_to_many :category
end

class Category < ActiveRecord::Base
has_and_belongs_to_many :recipes
end

==Conclusion==

CSC/ECE 517 Fall 2012/ch1b 1w63 dv

2012-10-03T20:28:25Z

Dthomas:

'''Active Records'''

==Introduction==

The ''Active Record pattern'' is a Design pattern in Software Engineering which deals with the approach to store and access data in a database. The interface of an object conforming to this pattern would contain functions to perform operations like Insert, Read, Update, and Delete. The Object will have properties that correspond to the columns in the underlying database table. This pattern is realised through ORM (Object-Relational Mapping) libraries in Programming languages.

''ActiveRecord'' is a module for Ruby that can be used for ORM. Thus, it is obvious that ActiveRecord will form a part of the Model in an MVC application developed in Ruby. The rest of the article discusses ActiveRecord that is the Ruby module for implementing the Active Record pattern.

The ActiveRecord module insulates the developer from the need to use SQL in most cases. Internally, It will perform queries on the database which corresponds to the method invoked on the object. This module is compatible with most database systems (most used ones like MySQL, PostgreSQL and SQLite). Moreover, regardless of which database system the developer uses, the Active Record method format always remains the same.

==Naming==

The ActiveRecord module uses a convention for naming classes, tables and fields so that the amount of configuration needed to get the functionality working is minimal. There are naming conventions on file naming, class naming, table naming etc.

===Reserved names and Attributes===

Certain names are reserved and should not be used (even in the model as attributes). Some of them are listed below:

*lock_version.
*type - This is only used when you have single table inheritance and must contain a class name.
*id - Reserved for primary keys.
*table_name_count - Reserved for counter cache.
*position - Reserved for acts_as_list.
*parent_id - Reserved for acts_as_tree.
*lft - Reserved for acts_as_nested_set.
*rgt - Reserved for acts_as_nested_set.
*quote - Method in ActiveRecord::Base which is used to quote SQL.
*template.

===Class Naming===

ActiveRecord classes are named in singular form. e.g User

===Table Naming===

Tables for ActiveRecord objects are named in plural form by default. e.g. Users
This naming convention can be circumvented by using below:

a) Set use_pluralization
In the environment.rb file we can specify
ActiveRecord::Base.use_pluralization = false.

This will disable pluralization for all ActiveRecord objects.

b.) Use set_table_name
You can call set_table_name to specify a custom table name for a particular model.
For example:
class User < ActiveRecord::Base
set_table_name 'user'
end

== CRUD ==

CRUD stands for 'Create', 'Read' , 'Update' and 'Delete'. These are the four basic operations which are generally performed on tables in a database. The ActiveRecord module provides predefined methods for the basic CRUD operations for the model.

3.1 Create

A new record can be created in the table by invoking the “save” function on the model object whose record has to be created in the database. ActiveRecord will use the Object's attributes as the field values for the record. The data is not persisted in the database until we call the save function.

@user = User.new
@user.name = “abc”
@user.age = 23
@user.save #returns a boolean indicating whether the save was successful or not (whether a new record was created or not)

ActiveRecord provides another convinient way to create a new record without creating instantiating the model explicitly and then using “save”. To do this, we use the 'create' function. By default the primary key used in the table is “id” which is generated automatically.

User.create(:name=>”xyz”, :age=”23”)

3.2 Read

A record can be read from the table by using the various functions like “find” (find the model record by specifying a value used in its primary key), “where”, “all” , “first” and “last”. All these functions instantiate a new Object for the model and populate its attributes using the fields of the record.

@user_first = User.first #Finds and returns the 1st User from the table
@user_last = User.last #Finds and returns the Last User from the table
@all_users = User.all # Returns all the Users from the table
@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_other_user = User.where(:name=>”abc”) #Finds and returns the user whose “name” is “abc” .

Dynamic Finders

Some of the most common searchs performed on databases are to return the rows where a column matches a given value. In many other languages and frameworks, we would generally need to construct SQL queries to perform these searches. ActiveRecord uses Ruby’s dynamic power to do this for us.
[edit] Connecting to the Database
For example, our User model has attributes such as name and age. We can use these names in finder methods to return rows where the corresponding columns match some value:
@my_user = User.find_by_name(“abc”)
@my_user = User.find_by_age(15)

3.3 Update

A record in the table corresponding to a given model instance can be Updated by using the function “save”.

@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_user.name = “test”
@my_user.save

Moreover, we can combine the functions of reading a row and updating it using the class methods update and update_all. The update method takes an id parameter and a set of attributes. It fetches the corresponding row, updates the given attributes, saves the result to the database, and returns the model object.

@my_user = User.update(1, :name=”test3”)
@result = User.update_all(“age= age+1”)

3.4 Delete

A record can be deleted from the table by invoking the “destroy” functionality on the instance of the object.
The destroy instance method deletes from the database the row corresponding to a particular model object. It then freezes the contents of that object, preventing future changes to the attributes.

@my_user = User.find(5) #Finds and returns the record from the users table whose id = 5
@my_user.destroy # deletes the record corresponding to the user with id = 5 from the table
# ... my_user is now frozen

It also has two class-level methods, delete and delete_all. The delete method takes a single id or an array of ids and deletes the corresponding row(s) in the underlying table. delete_all deletes rows matching a given condition.
User.delete(1)
User.delete([2,3,4,5])
User.delete_all(["age < ?" , 18])

The “delete” methods bypass the ActiveRecord callback and validation functions that may be defined for the model class, while the “destroy” methods ensure that they are all invoked. Hence, it is better to use the “destroy” methods as it ensures that our database is as per the business rules defined in the model.

==Connecting to the Database==

The ActiveRecord connection adapter is meant to wrap and abstract away the underlying driver specific to database, and is meant to provide an interface which is common for database tasks such as creating and destroying databases, modifying tables, updating, deleting, and inserting data, managing transactions and running queries . The connection adapter is normally used internally by ActiveRecord but can be used without the help of ActiveRecord models as well.

A connection adapter can be obtained in the following manner:

connection = Category.connection
object = Category.find(1)
connection = object.connection

Most applications connect to only one database which is defined in the database.yml file. In such a scenario every class which inherits from ActiveRecord::Base will be using the same connection. But in some special cases the application may also connect to a secondary database. That is the case in which some ActiveRecord classes connect to a secondary database. In such cases extra care need to take so that every class asks for a connection from the right database.

Rails generally opens several connections at once, and these connections are managed in a pool. Each connection adapter object forms a single connection to some database. Connections can run only one SQL statement at a time, so generally one connection is opened per thread. When a job needs a connection to database, it checks out one of the pool which is returned when it finishes for use by another task.

===Running Low-Level Queries===

ActiveRecord “model” objects are returned by ost calls in the standard ActiveRecord API. There might be cases in which you want to bypass the overhead involved in creating full ActiveRecord objects, or maybe want to query data that does not have a corresponding ActiveRecord class. SQL queries can be written using the connection adapter's low-level query methods.

In this first example, we get the “category_name” value from a single row in our “categoriess” table. If we only need the category name, we can grab the connection object and use the select_value method as shown below:

connection = Category.connection
category_name = connection.select_value("SELECT name FROM categories WHERE id=1")
# => "Football"

==Migrations==

Migrations help to version the various changes made to databases. It also allow developers to track a set of changes made to production or development databases and to rollback to a previous version if needed.

=== Building a Migration ===
You can either build the migration on its own using

<pre>ruby script/generate migration User</pre>

Specific commands can be written afterwards to create custom SQL. A model can also be created that comes with the migration.

<pre>ruby script/generate model User name:string user_id:integer</pre>

The migration will generate a couple of new files under the "db" directory. The contents of such a generate file are as follows:

# 9889904091223123_create_user.rb
class CreateUsers < ActiveRecord::Migration
def self.up
create_table :users do |t|
t.string :name
t.integer :user_id
t.timestamps
end
end

def self.down
drop_table :users
end
end

The "9889904091223123" at the beginning of the file-name is the timestamp. The timestamps will be different depending on the times of creation or modification the database. This is helpful to rollback to a previous state if needed. This way the developer need not remember how the current state is reached and how can he go back to a previous state.

The ''self.up'' from the previous code snippet creates the User table and add the columns. The ''self.down'' method is used to drop the table and to remove all the contents. The ''self.up'' and ''self.down'' methods are necessary to keep the database consistent after a rollback.

Rails adds an additional column called the "timestamps" to keep track of when each row was added. Rails also creates a primary key of the form "model_name"_id which increments automatically every time a row is added.

Different types of datatypes can be used with ActiveRecord. Some of the most commonly used ones are:

* integer
* string
* text
* boolean
* references
* decimal
* timestamp

The migration file can be written into the database using the following command:

<pre>rake db:migrate</pre>

The above command will create the table and the various columns. This command can be used to migrate many files in one go. Connections between tables in the database can be introduced using references in the model.

==Associations==

Associations are used to connect 2 models. The association is used to describe the role of relations that models are having with each other. ActiveRecord associations can be used to describe one-to-one (1:1), one-to-many (1:n) and many-to-many (n:m) relationships between models. Associations are used to make common operations simpler and easier in your code. Rails supports six types of associations:

* belongs_to
* has_one
* has_many
* has_many :through
* has_one :through
* has_and_belongs_to_many

''belongs_to'' and ''has_one'' form a one-to-one relationship. ''has_one :through'' is a different way to create a one-to-one relationship. ''has_many'' and ''belongs_to'' form a one-to-many relation. ''has_and_belongs_to_many'' or an alternative way ''has_many :through'' to create a many-to-many relationship.

===belongs_to Association===

A ''belongs_to'' association sets up a one-to-one connection with another model, such that each instance of the declaring model “belongs to” one instance of the other model. A ''belongs_to'' association can be used to setup a one-to-one or one-to-many relationship with other models. For example, consider a cookbook with recipes and categories such that each recipe "belongs to" a particular category.

class Recipe < ActiveRecord::Base
belongs_to :category
end

===has_one Association===

A ''has_one'' association is used to set up a one-to-one connection with another model such that each instance of a model contains one instance of another model. For example if we have two models User and Account, and each User has a single account, then we can use "has_one" to indicate the relationship between the two models.

class User < ActiveRecord::Base
has_one: account
end

===has_many Association===

A ''has_many'' association is used to set up a one-to-may association with other models such that each instance has zero or more instances of another model. In the cookbook example, one category can have many recipes.

class Category < ActiveRecord::Base
has_many: recipes
end

===has_many :through Association===

A ''has_many :through'' model is used to setup a many-to-many association with another model ''through'' a third model. In this case, the instance of a model can be connected to many instances of another model by proceeding through a third model. For example, consider a medical practice where patients make appointments to see physicians.

class Physician < ActiveRecord::Base
has_many :appointments
has_many :patients, :through => :appointments
end

class Appointment < ActiveRecord::Base
belongs_to :physician
belongs_to :patient
end

class Patient < ActiveRecord::Base
has_many :appointments
has_many :physicians, :through => :appointments
end

===has_one :through Association===

A ''has_one :through'' model is used to setup a one-to-one connection with another model ''through'' a third model. In this case, the instance of a model can be connected to one instance of another model through a third model. For example, each client has one account, and each account has one account history.

class Client < ActiveRecord::Base
has_one :account
has_one :account_history, :through => :account
end

class Account < ActiveRecord::Base
belongs_to :client
has_one :account_history
end

class AccountHistory < ActiveRecord::Base
belongs_to :account
end